#operating system design

[mentalisttraceur-software]

Process "File" Descriptors (PID FD)

Linux in recent years has fixed some of the biggest annoyances I had with the UNIX model, and I'm feeling rather happy about this. One example of this is with how you can refer to processes.

With traditional UNIX APIs, you can never signal a process without a PID reuse race condition. The problem is somewhat inherent to the model: a PID is just an integer reference to a kernel-level Thing, but crucially, that Thing is not being passed around between processes. If I tell you the PID 42, that's not qualitatively different from you randomly picking the number 42 - either way, you can try to signal whichever process currently has PID 42. You might not have permissions to signal the current resident of that PID for other reasons, but you don't need a prior relationship to signal it - you don't need to receive any capability object from anyone. I haven't given you something tied to any process, I just gave you a number, and in the traditional UNIX model there's nothing better (more intrinsically tied to a single process) to pass around.

Compare file descriptors. An FD is also "just" an integer, but having that integer is worthless unless you also have the kernel-level Thing shared with your process. Even if I give you the number 2, you can't write to my stderr - I have to share my stderr, for example by inheritance or by passing it through a UNIX domain socket.

This difference is subtle but technically crucial for eliminating race conditions: since an FD has to be shared from one process to another through the kernel, and the underlying "file" descriptor/description is managed and owned by the kernel, when you later use the FD in a system call, the kernel can always know a fully unambiguous identity of which Thing you were referring to. If you just told it 42 in some global ID space, well do you mean the thing currently at index 42? Or did you really think you were referring to a thing which existed some time ago at index 42? That information has been lost, or at least isn't being conveyed. But if you told it 42 in your file descriptor table, now that can hold more information, so it can remember precisely which thing your 42 was referring to when you last received or changed it.

So how did Linux fix this?

We now have "PID FDs", which are magic FDs referring to processes. Processes "as a file", in a more true and complete way than the older procfs stuff.

Unlike PIDs, which by design can only travel around like raw numbers divorced from the true identity information they gesture at, a PID FD can never lose track of the process it refers to: if that process dies, its PID FD will forever "know" that its dead even if the PID is reused (because when that process yields to or is preempted by the kernel for the last time, the kernel remembers that there's a PID FD for it which needs to be updated, and even if the kernel recycles the PID it knows not to update that PID FD to point to the new process).

And we now have a system calls which let us

create a new process (or thread!) and have that return a PID FD instead of a PID (or "TID" in the case of threads, but in the Linux kernel those are basically the same thing)

signal a PID FD instead of the traditional PID (or TID), and

wait on a PID FD (in fact, the FDs just plug into the normal poll, select, or epoll system calls, so waiting on process or thread status can just hook into all existing event loops).

The one crucial thing is that we must not forget that these guarantees only start once we've gotten the PID FD: in particular, if you get a PID FD from a PID instead of getting the PID FD directly, you still need to check once if the PID hasn't gotten invalidated between when you got the PID and when you got the PID FD. This is something you can only do as the direct parent or (sub)reaper of the PID in question, because then you can use the traditional "wait" system call on the PID after getting the PID FD - if getting the PID FD succeeded but then the wait reports that the PID exited, that means you could've gotten a PID reuse by the time you got the PID FD from the PID.

So ideally, we just don't deal in PIDs at all anymore. When we create a process, we can get its PID FD atomically with the creation. (If we can't do that, if we're creating the process and then getting the PID FD with a separate system call, we do the wait check described in the last paragraph after getting the PID.) Then we pass the PID FD around - this might superficially look like passing an integer around, for example if you need to use the FD number in a shell redirection or communicate it to a child process, but underneath that the actual FD would be getting passed around either by inheritance, UNIX domain sockets, or whatever other means are available.

I've been wanting this for years. I've spent a lot of mental cycles trying to figure out the most elegantly minimal and composable primitives which could work around various problems in service supervision and automatic restarts in the face of random processes dying or getting killed. Those edge cases are now much more simple to totally cover by using the PID FD stuff, at least so long as we're willing to lose compatibility with the BSDs and other Unix-y systems until they catch up. Which, honestly, is an increasingly appealing proposition as Linux has been adding good stuff like this lately.

P.S. On that note, another thing we now have, is the ability to reach into a process (if we have its PID FD and have the same relationship to it (or elevated privileges) that we'd need to hook into it with debugger system calls) and copy a file descriptor out of it. This means we no longer need a child process to have been coded to share it with us through a domain socket or by forking or exec-chaining into a process we specify. Accessing file descriptors in already-running processes enables some neat monkey-patching and composition, and it's simpler to implement than the UNIX domain socket trick. I remember being frustrated about 7 years ago by the lack of ability to do exactly this - reach down into a child process (for example into a shell script, which has no built-in ability to access the "ancillary data" of a socket) and pluck a file descriptor out of it.

[nitzukai]

some scrollbars.

[easternmind]

- Lessons in Japanese Game Design #3 -

Aside from what is easily one of the most stylish presentations in a videogame from the 1990s, one of Front Mission 3′s truly remarkable features consists of an in-game internet browser running within a simulated operating system. Expanding on the concept first rehearsed by Squaresoft in the second entry of the series, this massive component of the game can be accessed during breaks between missions, offering the player access to a number of pages for government entities, enterprises or other institutions. Access to additional websites is permitted as the game progresses.

This inventive solution offers unparalleled insight into the fictional universe of the game, in that it paints a very granular image of a futuristic society, its economy, geopolitical environment and zeitgeist. The information contained in these pages comes in the form of of bulletin boards, institutional updates, press releases, top secret data volumes, or the occasionally amusing curiosity. The user interface itself permits some degree of customisation, such as the option to disable the startup animation - itself a gorgeous visual relic that one hardly tires of watching - or changing the wallpaper pattern. Certain websites will also provide the option to download content, including images which can be set as said wallpaper. Other resources require secure access via the insertion of a password. Moreover, the player will have access to an email inbox to both read and send messages as may be relevant to advance onto the next stage.

Make-believe computer front-ends in video games were no novelty by the year 1999. Several sci-fi adventure, role-playing or action games allowed access to often very believable GUIs for the purpose of, for instance, retrieving precious clues or, say, disabling security locks. Aside from Front Mission 2, Final Fantasy VIII fans will likely remember the exciting Balamb Garden Network terminal. Kowloon's Gate's unforgettable Kownloonet terminals gave access to an email inbox, an all-important database and various other quotidian curiosities that helped fleshing out its mystic sci-fi narrative. The Japanese were privileged to have a complete version of Ace Combat 3: Electrosphere with access to an in-game encyclopedia the player could browse, as well as additional story elements told through email exchanges between the characters. 

That same year, the western Tactical RPG sensation from the West, Jagged Alliance 2, dabbled in this idea. Only three short years later, CyberConnect2′s singular .hack series was designed around the idea of a MMORPG that plays offline, allowing for countless interactions with non-playable characters which simulate messaging formats of the era. Other notable examples include the 2chan boards in Steins;Gate or, inescapably, the passionate love letter to web 1.0 that is Hypnospace Outlaw.

Generally, all of the aforementioned titles are either entirely dedicated on a similar concept, or merely resorted to it as a secondary, amusing distraction. Front Mission 3 pioneered the use of this device as a storytelling technique. No other game of the time, at least to my knowledge, had dedicated so much of its resources to producing a credible replica of design and functionality of the internet as it once existed.

[pixelglade]

For my first tumblr post from my new Linux partition, I want to share the changes I made to the user interface.

I changed the start menu icon to a Mew vector I made and changed the operating system logo on startup to a coloured version of the same.

For the start menu icon, be sure to look at the bottom left to see it in action next to the Menu button.

[industryhbo]

ppl need to stop making jokes about how it's so hard for "girls" to manage money/work/healthy lifestyle. because it's not a girl thing.

[purgemarchlockdown]

Gripping my table- Every day I get closer to writing a post about about fandom perception in milgram and how it gets to me that people are unwilling to reflect on the deeper themes of this work and how it handles the concepts of punishment and trauma and every day I have to remind myself that the reason Why I'm not writing that post is because I simply do not have the time, want, or are in the mood to write that post.

[clueless1995]

once again going to complain about how annoying it is to academically critique disney theme parks because everyone thinks you’re like a banksy dismaland escape from tomorrow “people having fun at disneyland are sheeple” edgelord like no you don’t understand i hate those guys Too their takes are immature and reductive

[frkfw]

An instant messenger application/program from a personal operating system concept.

[unproduciblesmackdown]

main idiosyncratic detail about LOS (besides that, in fact, all formal & informal reference to them alike is now LOS w/o the -307, as it understandably went in dialogue from the start but you'd think their name on the door / nameplate would be in full/official, so guess "LOS," in fact, is) is this portrait on their desk

small screencap, didn't think to try fullscreening it at the time, preoccupied making 6am audio posts you know how it is, but the gist is there. supposing this could be some character/s that's actually previously known, but if they're so Especial as to be the sole pic around you might think LOS would've also been there, making an appearance. maybe w/this kid they were such a guiding counseling success that it's like ooh let's get a pic w/said guidance counselor, who keeps it on their guidance counselor desk. but it looks very Family Portrait (+ guidance counselor? as the main alternate theory) & that's just kind of The Convention like oh if someone has this nondecorative framed photo on their work desk it's their family, honestly i'm presuming LOS hit it off w/someone & that is a delight to me

[collgeruledzebra]

head in my hands hal 9000 is not a robot... he's an artificial intelligence...

[uraniumglassgirl]

I love aokiji from one piece. Hes my little swimbly bimbly

[nitzukai]

clicky click.

[toytulini]

hit tag limit on the last post cos i started talking about roller coasters again 😔

[guandanarian]

Congratulations, you broke it!

[mtsodie]

hum ... i should design ocs based off operating systems ...

[experimental-daydream]

Words cannot describe how frustrated I am that life responds to disabled people by making their life harder and then pretending that it’s “assistant resources”.

If I am unable to spend multiple hours a day prepping, cooking and cleaning for meals why am I only allowed to buy things that need to be prepped and cooked?

(Also, I make minimum wage and it is apparently “too much” money so I don’t even qualify for benefits anyway.)