#i just can't manage to write well on PC for some reason
Explore tagged Tumblr posts
mezzmerizd · 12 days ago
Text
Oh my fucking god i just realised that if i switch to a new phone literally ALL of my writing will be gone,,,
^^^ uses samsung notes like a goddamn lunatic
4 notes · View notes
river-taxbird · 11 months ago
Text
Have YOU got an old Windows PC Microsoft has told you can't run Windows 11? It's time to give it a new life!
How to install Windows 11 on unsupported PC Hardware using Rufus. You can also disable some other Windows 11 bullshit like data harvesting and needing a Microsoft account.
It has been in the news a lot lately that Windows 11 isn't allowed to be installed on PCs without certain requirements, including the TPM 2.0, a chip that was only included in PCs made in 2018 or later. This means that once Windows 10 stops receiving security updates, those PCs will not be able to (officially) run a safe, updated version of Windows anymore. This has led to an estimated 240 million PCs bound for the landfill. Thanks Microsoft! I get you don't want to be seen as the insecure one, but creating this much waste can't be the solution.
(I know nerds, Linux is a thing. I love you but we are not having that conversation. If you want to use Linux on an old PC you are already doing it and you don't need to tell me about it. People need Windows for all sorts of reasons that Linux won't cut.)
So lately I have been helping some under privileged teens get set up with PCs. Their school was giving away their old lab computers, and these kids would usually have no chance to afford even a basic computer. They had their hard drives pulled so I have been setting them up with SSDs, but the question was, what to do about the operating system? So I looked into it and I found out there IS actually a way to bypass Microsoft's system requirement and put Windows 11 on PCs as old as 2010.
You will need: Rufus: An open source ISO burning tool.
A Windows 11 ISO: Available from Microsoft.
A USB Flash Drive, at least 16GB.
A working PC to make the ISO, and a PC from 2018 or older you want to install Windows 11 on.
Here is the guide I used, but I will put it in my own words as well.
Download your Windows 11 ISO, and plug in your USB drive. It will be erased, so don't have anything valuable on it. Run Rufus, select your USB drive in the Device window, and select your Windows 11 ISO with the Select button. (There is supposed to be a feature in Rufus to download your ISO but I couldn't get it to work.?
Tumblr media
Choose standard windows installation, and follow the screenshot for your settings. Once you are done that, press Start, and then the magic happens. Another window pops up allowing you to remove the system requirements, the need for a microsoft account, and turn off data collecting. Just click the options you want, and press ok to write your iso to a drive.
Tumblr media
From there you just need to use the USB drive to install windows. I won't go into details here, but here are some resources if you don't know how to do it.
Boot your PC from a USB Drive
Install Windows 11 from USB Drive
If you had a licensed copy of Windows 10, Windows 11 will already be licensed. If you don't, then perhaps you can use some kind of... Activation Scripts for Microsoft software, that will allow you to activate them. Of course I cannot link such tools here. So there you go, now you can save a PC made from before 2018 from the landfill, and maybe give it to a deserving teen in the process. The more we can extend the lives of technology and keep it out of the trash, the better.
Additional note: This removes the requirement for having 4GB Minimum of RAM, but I think that requirement should honestly be higher. Windows 11 will be unusable slow on any system with below 8GB of RAM. 8GB is the minimum I think you should have before trying this but it still really not enough for modern use outside of light web and office work. I wouldn't recommend trying this on anything with 4GB or less. I am honestly shocked they are still selling brand new Windows 11 PCs with 4GB of ram. If you're not sure how much RAM you have, you can find out in the performance tab of Task Manager in Windows, if you click the More Details icon on the bottom right. If you don't have enough, RAM for old systems is super cheap and widely available so it would definitely be worth upgrading if you have a ram starved machine you'd like to give a new life.
Tumblr media
703 notes · View notes
jester089 · 1 year ago
Note
Hey, hey! Listen, I had an idea.
have a theory in my head that all the characters are alive in the real life, which means they have a chance to get out. so, what would happen if the reader, for example, died in the real life (for example, put on a headset and... I don't know, shot themself..?Like, their body died, but their consciousness managed to end up in the circus), which means that they couldn't refuse the digital circus, because they simply don't have a physical body in the real world.
how would characters react(everyone decided to run away, but I would like to see the reaction of kinger , jax and pomni) if they managed to find a way out (exit), and when they were about to leave, their s/o said that they can't leave it and they have to stay here forever... like, just stay alone in an unknown place forever. but the reader says that everything is fine, and they will love and remember them all all the time? I just want it look little angst yk🫢 ( I prefer romantic)
I can't speak English good, so sorry for translating Ad I done this with translations
Thank you!!
Left behind
I was going to go to bed but then I saw this request. And I just can't keep my lovely readers waiting. Thank you for the request! And don't worry about your English it was perfectly legible. Plus my friend on discord have made me good at deciphering the most unreadable of things. So while reading your request I had some personal ideas/additions. So I'll, write what you wanted with a little of my ideas. You requested from me so I'm going to put my spin on it.
Kinger
You had grown close with Kinger bonding over both being borderline insane, but not abstracted. When a way out was found he didn't want to leave you behind. NO, he wouldn't leave you behind. You had to force him out of the digital realm so that he could live his life. And it hurt, being trapped in this hollow and ghostly version of the place you knew so well. But at least he could move on with his life. Even if it's without you. When the way out was found Caine disappeared for some unknown reason. So you are well and truly alone. No way of escaping, or dying. The only real way to get away from it all is to lose your mind to the point where you basically aren't human anymore. Kinger missed you back in the real world. He tried to move on but it was hard, knowing that your still trapped in that circus. So he went back to put the headset on again. If their's a set way out now he could visit you! But it didn't work. The pc, the vr goggles, the headset. They were gone... You were gone. So no one's coming to help you.
Jax
When the exit was found he tried so hard to just force you through it. (Like that scene from wreck it ralph where he tries to get Penelope out.) But as to be expected it didn't work. He was so stressed, his mind going over so many different outcomes. "If I leave will they still be here?", "I can't just leave them. After all we've been through. Fuck that. HOLY SHIT! I CAN CURSE AGAIN!!." Surprisingly he stayed with you. He stayed for around a month. But the ghost town of a circus clearly started to get to you two. And after that amount of time he started to care less and less. So he left you a note, then left. Didn't even say it to your face. And back in the real world he'll think about you every now and again. But he made it out of that hell. It's not his fault you didn't.
Pomni
Their is a 100% chance she's going to have a breakdown. She stayed determined to find an exit the entire time she was there. And you supported her every step of the way! And she did it! SHE FOUND AN EXIT! And exit... you can't use... She would freak out and be torn between leaving and staying. She gives you one more hug and kiss and promises she'll be back for you. She'll find a way to get you out from the outside! You hold out hope knowing your lovely is going to get you out of this place. You start writing down the days you've lived in a diary till your S/O gets you out. First 6 months are boring but you hold out hope. You know she'll get you out. You trust her. It's been a year. Maybe time works differently in the digital circus and the real world. Two Years now. Still not gonna give up on her cause she isn't going to give up on me!
ˢʰᵉ'ˢ ⁿᵒᵗ ᶜᵒᵐⁱⁿᵍ ᵇᵃᶜᵏ
ᴵˢ ˢʰᵉ
(Sorry this is so short. I couldn't come up with much and I'm really tired. But I feel like I made at least some 5/10 angst. Hope you enjoyed it!)
xoxo, Jester
459 notes · View notes
bhaalble · 1 year ago
Text
Weeks of very repetitive Gale jokes have made me a very aggro defender but I do still very much think they needed a different way to handle his response to the player having developing relationships with other party members.
Here's my thing. Its not a writing flaw that he's monogamous, or at least set on a monogamous relationship after his experience with Mystra. Its not a writing flaw that his romance flags develop pretty rapidly; after years with a very emotionally unavailable goddess and with the high stress scenario they're in I think it is very understandable that he would latch onto someone who not only shows him positive attention but is helping him manage an extremely frightening condition. Its also not necessarily a writing flaw that he's kind of a manipulative dickhead if your relationship is progressing further with someone else.
It just needed to be more confidently engaged with as a flaw.
Because here's the thing the passive aggressive sadboy act is certainly not BENEATH Gale. He tends not to handle losing very well. He doesn't have much experience managing feelings of inadequacy and self-doubt. He also just got out of an incredibly lopsided and uneven relationship where one of the things that continued to trouble him was feeling like a lower priority, like this person who was the center of his world saw him as just a nice accessory to hers. In light of all that I think its very understandable that he would lash out if he felt like the object of his affections was showing more preferential treatment to someone else, even if they were't properly in a relationshp yet. Its a self-defensive measure, not unlike Shadowheart writing you off for most of your early interactions or Astarion fully manipulating you with seduction during the first stages of your romance. Every companion is wrestling (and usually wrestling poorly) with the baggage of their previous situation and it will come heavily to bear on their relationship with Tav.
My primary issue is that outside of ending the relationship or an offhand comment about how you didn't know he felt this way (which leads to more of a pity party about how he just isnt enough for Tav) you aren't really able to push back on Gale's reasoning. You can sever the relationship, terminate the flirting stage, or you can just affirm him by choosing him. You can't really communicate hey. I care about you. Why are you speaking to me like I did this to hurt you on purpose. Even if he storms off in that conversation I think there very much was potential there to have some genuinely compelling character growth for Gale, with him confronting how many foundational issues there were in his dynamic with Mystra that he's unthinkingly putting on Tav. Tav would be given the opportunity to recognize where their partner is at and either draw a hard boundary, or extend a bit of grace depending on the feelings of that character and the player.
Again none of this required Gale to have a less shitty reaction to the scenario the point wouldn't be to scrub away conflict. Just to provide some additional opportunities to challenge your friend/LI and use it to develop both them and your PC
133 notes · View notes
ryuichirou · 4 months ago
Text
Replies
Some replies! One about drawing, another one about Ortho, and then some other TWST ones.
Anonymous asked:
Hello Ryu! I hope you and Katsu are doing well. I wanted to ask you: how do you manage to post daily? what motivates you? I just admire how you manage to post quality content every day, I enjoy everything so much! and another question: how do you fight hand pain? I do draw daily, but I end up with hand/wrist cramps and it's worrying me, what would you recommend? Amazing art as always!
Hi Anon! Thank you so much for your kind words.
I actually talked about our motivation and the ungodly amount of sketches (lol) that I produce here! But in short, I’ve always loved drawing, and I spend the majority of my time drawing. Not everything that we post daily is new stuff: we have a big backlog of sketches, and Katsu helps me to manage it. Since I can focus on drawing and Katsu takes care of posting, it’s actually feels easier.
As for the second question, the most important main thing is PLEASE don’t overwork your hand! The moment you feel wrist cramps, take a break immediately, or if the pain is too much, better stop for the day, even if you really want to continue :( I am lucky to have pretty good stamina when it comes to this, but this is also partially because I stop immediately if I start to feel tired, plus never draw after my lengthy timeslot dedicated to drawing is over.
Here are a couple of more things that could possibly help:
Pay attention to the way you move your arm when you draw. Prioritise moving your elbow and using your entire arm instead of just moving your hand and wrist. Also try not to squeeze your pen too much, as it tends to make muscles tenser, at least in my experience.
Take short breaks every once in a while to stretch and do exercises for your wrist. These work nicely, but I also love to just… not do anything with my hand at all for like 5 minutes, to let muscles relax a bit.
If your muscles are tense after drawing, warming them up also makes it feel better. I just hold my wrist under warm water lol But there are better ways to do it.
Oh, and check if you do something else with your hand that might also put pressure on it during the day. If you use PC, for example, and sit in front of a desk, then your elbow should be on its surface (the reason for that is in my first point of this list), otherwise you strain your wrist muscles with that position. This is just an example, but it might help to find other reasons why your hand can be tired even before you start drawing. If not, that’s also fine! Drawing is hard enough to do that kind of damage on its own.
If nothing helps and you still feel cramps, you should of course talk to your doctor about it. Maybe they’ll have better advice than I do. Please be very careful with your hand, these muscles are not the ones that should feel slightly sore after a workout unlike a bunch of others, this strain (and pain) can do irreversible damage to your hand and should not be treated lightly. It’s better to take frequent breaks and draw later than to lose the ability to draw (and do other everyday things along with it) at all.
Take care of yourself, Anon <3
Anonymous asked:
wow a whole shroudcest ask post!!! kissing each of the askers consensual and platonically on the mouth, i love that y'all in this fandom. also agree, Ortho should use his beams as a treat when not allowed to join mature content that has first years, it makes me really sad for him!! Epel is super small and he gets to join????? sometimes he's even ''aged up'' (and stays short and small, i see that) lmao like what, Ortho can't be aged up too? the double standards are staggering
sometimes i assume he's just hard to draw, it's fair when mecha and armor are difficult for some people and i often see those two in won't draw lists in commissions, but also?? you don't have to draw when writing?? at least in writing, he should show up more, i'm doing my part at least in the orthoreader and orthooc tags on ao3. his crotch being bare is a feature, not a flaw.
(related to this post)
I am so happy and shocked that so many people wrote such long and lovely messages!! Everyone GETS IT…
Yeah, Epel also gets to join all the time, but somehow Ortho isn’t! This absolutely isn’t fair lol He would be super pissed about it.
I agree with you though that it is related to his design. Of course, the majority of people who complain about Ortho don’t even attempt to draw him as they aren’t artists, but I think a lot of aspects of his appearance are somewhat… “off-putting” to them? It feels like it’s a very strong word, but: his height, his “heels”, the waist and crotch parts of his gear, plus he isn’t very manly and ikemen-like, instead his appeal is that he is a cutie and sometimes a “kusogaki” type. This is why some people just don’t like him at all, and some people don’t like him and cover it up by saying “awww nooo he is just a baby, look at him, he is a kid right?”
I am happy you’re providing people with written Ortho-related content, Anon 💪 He would be so proud and grateful!
Anonymous asked:
Your chibi lillia just looks so adorable, he looks so chewable
Hehehe thank youuu!! Please chew on his chibi cheeks.
irregardlessly-tish asked:
Have you seen the new design that was announced? (Trying to keep it vague in case people don't know which character it is, I don't want to ruin the surprise lol) From an artist point of view, what do you think? I was kind of disappointed by it… I'm not sure if it's Yana's design or not, her characters have so much personality but this one looks just… so generic? Love his suit, the glasses are a great touch to represent this character's eyes (orlackthereof) but everything else feels so uninspired in my opinion. I know this is the pretty anime boy villain game but this one is just too much of a pretty anime boy and I hate this fluffy pretty anime boy hair...
I have! And I will spoil it so please don’t read my reply if you don’t want to know who it is lol
To be completely honest, I am not sure what to think yet; I kind of want to see the full design and also to see him in action. He does feel very different from Rollo and Fellow&Gidel, and maybe this is one of the reasons why he might feel underwhelming: those guys have very colourful designs and a lot of accessories, plus somewhat clearer silhouette (Rollo’s hat is iconic, Fellow also has a hat and fox ears, and that makes them very distinct right away), and the new guy… it’s hard to tell yet, but I think it’s kind of difficult to make a skeleton design work on the same level both in terms of his monochrome colour scheme and the fact that he is, in fact, a skeleton-like creature.
I know there are a lot of jokes about Gojo that people make (I am one of those people), and I think that some people aren’t as excited for the new guy because of that (the feeling that we’ve seen him around a lot already, because young!Gojo was everywhere very recently). But I do agree with you: the glasses are indeed a very good touch and a very smart addition design-wise.
I am also personally excited to see Undertaker vibes from him, but that’s personal lol
So yeah, I’m not sure for now. I really want to see more of him before making proper judgment. I absolutely see your point though. I think if he turns out to be a fun character, his design elements will also become fun and enjoyable, just in a different way.
15 notes · View notes
s-dei · 3 months ago
Text
Tagged by @thetentaclecommander !
Not that much of a writer, but I will answer what fits 'w'
When did you start writing?
I remember writing silly stories w/ my friend for lulz in the middle school. But well, many ppl were writing those cringe yet fun stories at the time, and I was fascinated by them till I became older. The first time I wrote something complete and shared it online was in 2019. I think since then I began to at least writing lil scenes, without aiming to finalize them tho.
Are there different themes or genres you enjoy reading than what you write?
Not reading much and not writing much to tell. But I think I can read anything except my squicks, while my writing is mostly about something happening between certain chars.
Is there a writer you want to emulate or get compared to often?
Nope, currently just lookin for my own style
Can you tell me a bit about your writing space?
I mostly write on my PC in G-docs. Sometimes I type lil things on the smartphone notes. They're both actually dumb, coz PC have too many distracting stuff, and the phone is just uncomfy with it's tiny keys.
What's your most effective way to muster up a muse?
Just to be a stupid dog over my stuff xD Well, honestly I haven't got a habit yet, and don't have a ways to warm myself up after a long pause. Those two times I managed to write something were "wow, there is and idea and I like it" and I was writing in the same day and finished it within a week or something. I think I have enough WIPs to have one done without a something like 'muse', just my pragmatic will, but I always can't find the right time (drawing gets in the way xD)
Are there any recurring themes in your writing? Do they surprise you?
hmmmm I think what I like is "what keeps people together?" It can be any type of relationship, but I enjoy scenes where something between chars is changing. Strangers to friends. One char realizing their feelings for another. A pair having the first time. Lovers to enemies, etc.
What is your reason for writing?
I may grumble, but writing is fun. It's another, uncomparable to drawing act of creation, with it's own pros and cons. Some things just go easier with writing.
How do you want to be thought about by your readers?
idk but i hope we can make a pack
What do you feel is your greatest strength as a writer?
Being niche and fked up enjoyer? :D
How do you feel about your own writing?
Honestly - I love it. Sometimes I do re-read my WIPs and think 'damn those are good, when will you bring your ass to finish them?' But even just re-reading cheers me up. Sometimes it's even fun, coz in the hard moments I often do write silly and it keeps my attention on it
Tumblr media
When you write, are you influenced by what others might enjoy reading, or do you write purely for yourself, or a mix of both?
Just following my heart and what I want.
Feel free to grab the questions! 👁👄👁
1 note · View note
stinklebug · 10 months ago
Text
since i'm not going to get around to actually writing a fic for the story with indigo for a while, i figured i may as well talk about it a bit!
i don't have everything figured out yet, this is just the stuff i currently have sorted
it takes place not long after the gang finished up dealing with the spectra doors & hub. with all of That out of the way, chai has to actually Do His Job, which is how indigo and him meet, of course
indigo is sort of the pseudo head of marketing until they sort out All Of That. it worked under mimosa for about a year until the gang came along, and it absolutely Hated her. so it's sort of thankful that they came along. but also it doesn't like having to work with chai, sooo not a total win.
it mostly doesn't like him because he's loud and Always plays its mom's songs whenever he's around it. despite the fact that indigo's told him Not To Do That. unfortunately, he can't really control what his player decides what to play and when it chooses to play it, much less when it deems it appropriate to play said music out loud.
chai doesn't hate indigo, but he also doesn't really like it. he's mostly just confused as to why it seems to hate music so much. the most he knows is that it despises one artist in particular: limeade. said artist is a Very famous rockstar who chai is a big fan of! indigo acts like she's the scum of the earth for some reason. he's asked it about why it hates her so much, but it refuses to explain, always just saying, "it's none of your fuckin' business."
the others don't really like indigo a whole lot. this is because they mostly just know about it through chai, who paints it out to be a big drama queen hater. their opinions change once they actually meet it, but that doesn't happen for a while
before then, indigo gets a flash drive which, unbeknownst to it, has a back-up version of SPECTRA on it. this back-up isn't as powerful as the regular version, due to it being on a hard drive and not an entire super computer, but is volatile in its own unique ways.
it only gets the flash drive because it was down in security (for reasons i have yet to figure out), and a TEC-78 dropped it. said TEC-78 proceeded to very quickly disappear. with no time to hunt them down, it just pocketed it and intended to give it to another unit, only to forget about it as indigo got caught up in the reason it was down in security for in the first place
it winds up getting curious, and plugs the drive into its laptop. but...nothing happens. the contents of it are all things that it doesn't know anything about, much less how to actually get any of it to operate. so it decides to stick to its original plan of giving it to the next TEC-78 it sees.
but indigo just...keeps forgetting to do it. to be fair, it's not down in security much, which is where the majority of those particular units are. and, when nothing bad seems to happen from one single flash drive going missing—no big alerts being like, "MISSING FLASH DRIVE, CONTENTS VERY IMPORTANT," or anyone getting fired—it figures it may as well just use the thing as extra storage.
what it Doesn't know is that spectra is Very Functional and Very Aware. the ai is Not happy about being in some random person's pc...so limited! so cramped! no hologram form! can't have shit on a basic, crappy laptop!
so indigo moves on with life, and spectra is just kind of There.
speaking of indigo's life, it's starting to get more and more hellish. chai keeps pissing it off, whether that be by just being kind of a dick or simply a nuisance. it's starting to get fed up with him, and things only get worse when they have to be around each other Constantly because of all the marketing and events he's in that it has to manage
after a particularly long day, it's So Pissed it just sort of starts ranting to itself. spectra, hearing it complain about chai, switches gears from being Just There (which throughout the whole thing with chai pissing indigo off, spectra starts subtly making its presence known, but indigo just sort of brushes it off thinking that its laptop is just doing Old Stupid Laptop Things) to "oh i can use this"
with it now influencing indigo through classic manipulation instead of mind control, spectra's world starts to open back up again.
indigo begins sabotaging chai's performances and starts befriending the gang, who are all a bit suspicious at first. but, eventually, they come around thanks to its charm and some shared interests. they also befriend chai, but it's less of a friendship and more of a "i hate you but i have to pretend to like you for Reasons"
chai starts to get isolated because of this (he doesn't want to hang out with indigo because it makes fun of him, under the guise of "i'm just messing around! they all tease you, so why can't i? we're friends after all, aren't we?")
which is great for indigo, because it means no one is around to stop it when, late one night while having a sleepover at the hideout, it plugs the flash drive into peppermint's pc. then, in the morning, before anyone wakes up, it unplugs spectra and sneaks out, leaving a note about having to go do something
now that there's a copy of spectra in peppermint's pc, it starts Causing Problems. meanwhile, the one with indigo starts convincing it to go down to the Big Computer and plug it back in. either that...or it convinces indigo to give spectra access to Itself. not sure yet!
that's all i really have for the moment. like i said, i don't have everything sorted. this is just sort of the Skeleton of the narrative.
5 notes · View notes
miloscat · 4 months ago
Text
[Review] Sonic Lost World (Wii U)
Tumblr media
Super Sonic Galaxy!
After Colours, Sonic Team went all out on Generations, refining the 3D boost formula successfully (I've played Generations before so it's not included in my current run through). This is now the era where Colours had somehow been successful and Sega were all in on supporting the Wii U... for the moment. Racing Transformed had special Gamepad modes, and soon Rise of Lyric and the Winter Olympics game would be exclusives for the system, and meanwhile the big new platformer from Sonic Team got a Wii U release and a 3DS companion game by Dimps [to be reviewed later]. In a couple of years time when the writing was on the wall for the Wii U, Sega would port Lost World to the PC, but it's a Wii U game through and through.
On its home platform, Lost World can only be played with the Gamepad. The Wisps, returning from Colours, are controlled with gyro and touchscreen [optionally, thank goodness], along with between-level minigames. Otherwise the second screen shows the usual HUD elements of time/lives/rings etc., your level progress, hints, active missions, and a half-baked inventory system. It's an attempt at dual-screen gaming but I can see how it could all be reworked or moved to a pause screen easily enough.
Tumblr media
Otherwise Lost World represents a complete departure from all prior Sonic games. They've totally rethought Sonic's physics, the controls, the whole gameplay paradigm, all in the name of... chasing Mario Galaxy's success, I guess. Now if you're challenging something so well-received, you need to do it well. Lost World doesn't.
The gravity gimmick is the headline feature, with 3D spaces often taking the form of long cylinders or small spheres that freely rotate as you move through them. 2D sequences have some measure of curving as well, but they're usually straightforward sidescrollers. To be perfectly honest I think the whole thing is implemented better here than in Galaxy itself as it doesn't push it too far, keeping it as mainly an aesthetic/structural feature. The returning Wisps remain gimmicky but they actually managed to make most of them work in 3D this time, whether it's old colours or new ideas. They're maybe a step up from their implementation in Colours itself and have less of a focus which also helps.
Tumblr media
As for Sonic's movement, he has a run button now, maybe for the first time? The fact that he doesn't conserve momentum at all feels odd but is probably for the best with the type of platforming required. A parkour system is supposed to complement this by letting you run up walls and grab ledges; this is a good platforming crutch but often will misfire, only making things awkward and interrupting flow instead of facilitating it. There's also a wallrunning mechanic and dedicated parkour stunt button, neither of which I ever found useful, intuitive, or fun.
Sonic can spin dash which is used maybe a few times, and there are for some reason two separate homing attack buttons: one is a kick and can't snap you to the ubiquitous bumper launchers, the other is chargeable and combo-able and the key to doing any damage to bosses. The game didn't adequately explain this so I got through the first world or so only using the kick, which made the first boss miserable and janky... but I thought that the game was just like that! In the end though it still was a bit like that.
Tumblr media
The game plays out one a Tiny Planet-esque landmass called Lost Hex, which is visually made up of floating hexagons. Well, it is in the map screen anyway, I don't think this really carries over into any levels... It consists of your typical platformer environments skinned over the same core gameplay: plain, desert, jungle, ice, lava, etc. but everything looks quite nice and unlike many other 3D Sonic games it runs consistently smoothly. Individual levels do a decent job varying the biomes, while some bonus levels get a bit more surreal and there's even a truncated boss rush themed after Sonic Team's Saturn classic Nights Into Dreams. The Wii U version also has some fun extras in the form of a short 2D level with a Yoshi's Story aesthetic that plays like a quite bland Yoshi's Island, and a Zelda level with a sort of hybrid Ocarina/Skyward Sword aesthetic that has a mini-open zone Hyrule Field and a sort of mini-dungeon that reskins Lost World's "tube interior" gameplay style. Modders have added these to the PC version so there's no reason to miss out.
The story is building off Colours, with the same writers returning with their weak jokes and shaky characterisation. This time there's more interaction between Eggman and Sonic as they're forced to team up; I like the concept although their dynamic plays out weirdly. Orbot and Cubot are a bit more fleshed out, and Tails is... participating. Amy and Knuckles exist, if only to call Sonic on Facetime twice to try and instil some stakes. Meanwhile the villain squad represent a new nadir for the franchise. The six Zeti have weird cartoonish designs and offensively stock personalities. For example, one of them is big and fat and likes eating and always talks about eating. Funny! Another one is the token girl; she is vain because she's a girl and get this, because she's a girl she's obsessed with her hair and makeup! Wow! Somebody get these guys a Bafta!
Tumblr media
For the first time since I think Adventure 2, the animals that can be rescued from defeating Badniks have a gameplay purpose, even if it's just unlocking the final act of each zone. But they make for a persistent counter that marks your progress and a reward for minigames. In a level your rewards are Red Ring collectibles that unlock more minigame stages (these got old for me about halfway through) and lives. You can quickly go through lives on some sections but the game also throws a lot of them at you, and the DLC stages are a quick and effective source of life and animal farming.
Omochao's missions are a nice bonus for doing various actions, but you can only hold three at a time and I spent half the game with the same uncleared set because I couldn't figure out how "ceiling running" worked or be bothered to replay older levels that uniquely had certain kinds of Wisps. These reward items that can be redeemed mid-level but aren't very useful, and again there's a limit of five. Get any more and the game tediously prompts you to discard it or send it to another player via the now defunct Miiverse feature. Other mission rewards consist of various cosmetics for the co-op feature, which I think is similar to the Mario Galaxy "younger sibling" type of "multiplayer". And speaking of this era of Nintendo, Lost World also has the patronising "skip ahead a checkpoint" prompt if you die too many times, which I only ever activated by accident thus cheating myself out of gameplay. Thanks game.
Tumblr media
The soundtrack by my new favourite person Tomoya Ohtani ranges from pleasant to "rivals Rooftop Run", which is high praise. I was bummed to discover that there is no new main vocal track this time as those have been highlights of his soundtracks, but Forces and later Frontiers make up for it by having tons of them.
Totally reworking your core gameplay concept is a big gamble which can either pay off (eg. Unleashed Day stages), or it can backfire (eg. Unleashed Night stages) [your mileage may vary, of course]. In the case of Lost World, on balance it's more the latter. There's certainly potential here but it's a shame to seemingly throw away the progress that had been made in getting to Generations. At least they picked that back up and hit a sweet spot for me with Forces, which makes Lost World a one-off that gets consigned to the "failed experiments" pile with the Storybook games. I guess over the long term the franchise is constantly reinventing itself and I respect that, but they can't all be winners. But before we dispense with Lost World completely, there's a 3DS version created by Dimps that I must investigate! Until then...!
1 note · View note
darkfeanix · 1 year ago
Text
Baldur's Gate 3 Thoughts, Post-Game
It's the weekend, so time for me to put my Baldur's Gate 3 thoughts to page. Literally, I typed this up in MS Word because I no longer trust Tumblr. Or, I guess I should say I trust Tumblr even less than I did, because who really trusts Tumblr these days?
But I digress. I really enjoyed the game. I would go so far as to say I loved it. It stands out even more now because I was reminded this morning that Phantom Liberty comes out in the near future, and it got me thinking again about how thoroughly disappointed I was with the ending of Cyberpunk 2077, and how Baldur's Gate 3 somehow managed to give me a hugely satisfying ending even if I didn't quite get the happy ending I hoped for.
More under the cut, for SPOILER reasons, including discussion of the Absolute's identity in the very first paragraph. Let's just say SPOILERS for the whole game; if you don't want to be SPOILED for anything, go no further.
So, the ending. Loved it. Loved the way it was designed (in part because I was able to bypass most of the fighting with Misty Step and Dimension Door). I was also really glad that the "Absolute" turned out to be the one pulling the strings the whole time. While it was a pretty fun twist that the three Chosen were secretly behind the Cult, I will admit to being a bit disappointed, as it felt like a bit of a letdown to go from "powerful goddess influencing the followers of other religions" to three power-hungry mortals playing with forces they didn't fully comprehend. So having it then turn out that the Elder Brain was working behind the scenes the whole time felt like the revelation that I had really been hoping for regarding the Absolute's identity.
As far as NPCs and their plot arcs go, I really loved all of them, but Shadowheart's was probably my favourite. Between the writing and Jennifer English's incredible performance, that was the one that just really spoke to me the most. I was so invested in helping her achieve her Dark Justiciar dream, and then my heart just broke for her as more and more layers of her backstory were peeled back. Her emotional journey all throughout really pulled me along and did not once let go.
Speaking of emotional journeys: Lae'zel. I mentioned in a previous post that I had intended to romance Karlach, and that my flirting and sleeping with Lae'zel was just done For Science. However, something happened that neither I, nor my PC, Meriam expected: Lae'zel caught feelings.
Tumblr media
"Tsk'va! No more. I can't bear it. I've torn flesh from monsters and men. I've laughed as they suffered. But you... I don't want to hurt you - I want to protect you. For you to protect me."
Oh, Lae'zel, my beloved.
Meriam returned those feelings with all her heart, even though that meant breaking Karlach's heart. I do have a minor nitpick there, which is that I ended up getting serious emotion whiplash from Karlach being heartbroken in one dialogue, and then going back to normal in the next, as though nothing had happened. I understand that even a game as vast as this one has limits, but I do wish there had been some kind of acknowledgement of it, even if only during the next conversation that occurred after the break-up.
But that's just one small thing out of a huge game that gave me so much agency in what I did with my character. Case in point:
Tumblr media
Meriam Goodflower in her final form.
I knew from a very early point in the game that if the option to become a mindflayer was ever presented, then Meriam would probably take it. However, I never expected it to be presented in a way that just worked so well in convincing her as a character that it was exactly what she wanted.
Tumblr media
"You can make only one move at a time. But the Netherbrain calculates every possible move at once. It knows what you will do, it knows everything you could possibly do. You cannot outmanoeuvre it. To defeat it, you would have to think like an illithid. Better yet, be one. Your mind is not capable of this. Mine is."
Ultimately, her choice had nothing to do with beating the Netherbrain. Meriam became an illithid because an entirely new way of perceiving and interpreting the world was presented to her. That would have always been my headcanon, but as I said, I didn't expect it to actually be there in the text for me to justify it.
What's more, none of her companions abandoned her. Even Lae'zel, a githyanki, still saw her for her she truly was under those tentacles. For a brief time, I thought maybe Meriam could have it all.
In the end, though, I'm glad that Lae'zel chose not to remained with her. As much as I wanted me first BG3 romance to have a happily ever after, I feel like having Lae'zel settle down with a mindflayer just wouldn't have been true to her character. And Meriam understood as well; it was enough for her just to see this when they parted ways:
Tumblr media Tumblr media
That smile made it all worth it for her.
And with that, I don't believe I have much more to say, at least for now. I hope in the future to actually write some fanfiction about Meriam's journey, and even some post-game stuff as well. In particular, there's a story to be told about Meriam and Astarion, post-game, both irrevocably changed and not necessarily for the better.
But that's for another time.
0 notes
kenzan-kiwami · 1 year ago
Text
NO i love it too!! livejournal itself was a bit before my time but i've done wordpress comment sections before lol
always made me so sad that kiryu doesn't find out about the HLA until after kashiwagi dies, and also that i never finished that part of the game when i played 3 because i had INTENSE gun boss memories from kiwami and i was convinced it would be the same in 3 for some reason??
full disclosure, i never played y4, but that's really interesting to know about. also really funny to me because i swear half of the things we know about kashiwagi come from things other people say about him (bad back anyone)
thinking about it now though i imagine kazama and dojima's families merging in the 90s and then eventually taking over the family himself would have brought a lot of capital into kashiwagi's hands that he could have put towards starting the HLA up - i seem to remember them saying kamuro castle was like a theme bar at some point before they repurposed it, which was why it was outfitted like a dungeon? but yakuza 3 is on my replay list for good reason lol
0 too, for kazama reasons... it would make sense he's not as popular as kashiwagi for those reasons especially, and in general i think the games seem to present him as kind of an asshole despite kiryu as the main protagonist being informed by things kazama said or did through every step of his life. i know a lot of people in this fandom love to meme on him being a shit dad, and i do find it really funny too until it starts to broach "i hate kazama unironically and i think he's completely terrible and irredeemable" fanonisation territory cause it's so disingenuous to his character. but i do think that mentality and the lack of in-depth kazama analysis propagate each other in a sort of feedback loop
hah, might have to open up my docs while it's quiet at work next week... he deserves a treat. and nishiki deserves to have a moment of being portrayed as competent for once by organising informal dinner for his bros. :) but i think i would need to go back and play some pokecir and real estate minigame before i feel comfy writing about it hehe
I HAVE thought about the pre-fight ficlet though. i'm not super confident in my ideas, but i love the thought of them coming out of an alley next to cafe alps and kashiwagi offering to buy majima a coffee before they beat the shit out of each other, cause he walks into the office looking like he got dragged backwards through a bush and i think (especially by that point in the game) kashiwagi can absolutely relate and sympathise. i think for me at the moment my biggest singular issue is stringing things together in a way that flows well & generally what i've written since i was a preteen has been very short scenes or moments that don't require a lot of thought towards pacing or intensive dialogue lol
but yeah, reuse ga gotoku is real lol. i dug up kashiwagi AND that giant-ass dragon from 7 in gaiden's files when i was looking for shishido, and idk maybe they use the dragon somewhere but DEFINITELY NOT WAGI <\3 them not re-rendering the orphanage pic is why i'm so crazy about what's going down with haruka in 8 because they redid her princess league scene in dragon engine and i'mmmmm very interested in what this might entail
the cap's actually from nanba's drink link! i think that scene from 7 is the only one i haven't managed to dig up yet in those big cutscene videos (because that's basically my only option if i want to watch 7 with jpn audio) but mobile makes it so hard to navigate sixteen hour uploads : ( Some Day when i'm at my own PC
i still just can't believe everyone who finished the game saw that scene and then only like three or four people started writing about them together LOL
(screenshotting the replies and posting because i feel weird replying from my main <\3 hope you don't mind the ping @startledpixel )
Tumblr media
i never even thought about it happening that way... haha... excuse me while i go scream in a corner for a couple of minutes
it ties in really well to the sort of recurring motif i see through his life that the only time he's allowed to be truly happy with himself and his place in the world is after his "death" and recovery. kazama is a very complicated character to think about when it comes to his motivations and his relationships with the people he's close with - i don't think he's very good at prioritising his emotional investments (nishiki being the obvious example, but also the way he doesn't send any letters to kiryu in jail until the last day before his parole), so to add an extra layer of tragedy to the whole thing i like to think that he hadn't been making as much time for kashiwagi towards the end than he maybe could have been. not in a premeditated disloyalty sense, of course, but he's got his hands more than full with the whole embezzling 10 billion yen from the tojo coffers gambit... kashiwagi being the way he is though would be all the more desperate for reconciliation, and to then not get it before kazama kicks it would be the icing on the shit cake
but yeah KNOWING adachi was in the building with everyone else must have been like reliving his second-worst nightmare... meeting this man he thought he'd be able to settle down with for the first time in sixty odd years but still constantly having to worry after him. i still adore no idling as an exploration of those feelings after the fact & i find myself coming back to it an awful lot as someone who doesn't generally read fiction more than once or twice unless i'm trying to find something specific (if you may allow me my nerd moment)
it's something i would love to explore more myself, but i don't really feel i have the means to do it in a way in which i'd enjoy the end product... but i suppose that's what commissions are for!
ANYWAY, apparently, everyone kiryu meets in his side story gives him some kind of reward, and i'm having A Time thinking about what he might get from kashiwagi. i'm trying not to set myself up to be disappointed by what happens, but there's a big part of me that hopes kashiwagi pulls "suzuki" to the side and leaves whoever else on the bar for a while so they get a chance to actually catch up. i think at this point both of them really need something like that, because i doubt there's any way kashiwagi didn't get the news that kiryu "died" in 2016
the other big thing that's got me physically shaking is the idea they might finally namedrop him. and uuhhh if they still let us karaoke at survive then i hope judgement gets its own cinematic. : )
8 notes · View notes
kazumaple · 3 years ago
Note
can i request a cute? maybe steamy? optional bias scenario where theyre gamers hehe ur writing is v cute btw! <3
cherry ♥ optional bias
description: gamer!au, highschool!au, enemies!au word count: approx 1.6k (1,597) warnings: mentions of food, cursing.
i'm glad you like it! thanks for the request.
(p.s: clicky switch enjoyers please don’t hate me/hj)
link to masterlist. click here to make a request!
Tumblr media
sighing in frustration, you gripped the steel bars of the door to the pc room, hoping to relieve some stress from the day through your favourite game.
it had been a rough day, with three tests to sit, and frankly, you didn't think you did particularly well on any of them, despite the hours of intense studying you had done in preparation.
the one thing that would make every thing worse was seeing him.
you prayed that for some reason he was busy, or was unable to attend today, but no. there he sat, eyes trained on the screen, focussed on winning his game. his usual smug smile gone, replaced with a serious scowl, you could only assume he was losing said game.
you took your usual seat, adjacent to his, and turned on the pc.
just as you did, he let out an anguished scream.
"can you shut up," you requested, "i'm not having the best day, and hearing your voice makes it a whole lot worse."
"the world doesn't revolve around you, so suck it." he retorted, not even sparing a glance at you.
the nerve on this guy. you thought.
"i can't deal with this today, just be a nice person for once, i'm sure everyone would appreciate it."
"excuse me?" he sat back in shock, looking at you for the first time, “you were the one telling me to shut up seconds ago, so don't try to lecture me about being nice."
just as you were about to say something in response, you stopped yourself, getting out your headset from your bag and choosing to use games as the outlet for your anger instead of him.
he was stupid anyway, there was no point in arguing.
rolling your eyes at him one last time, you focussed on the game.
---
the next day, you assumed it would be better.
and for a while, it was.
you had your favourite breakfast, picked out a hairstyle that you thought was cute, and managed to finish all of the homework you had the night before.
but, the universe chose violence today.
because when your teacher walked in that morning, they brought devastating news.
"students!" they shouted, "we have someone new today. his name is h/n."
at the sound of his name, your eyes perked immediately to the front of the classroom. usually, you didn't care about new students. but, this time, you had no choice. because there he was, in all his glory, l/n h/n. and for some reason, he looked disgustingly good in the school uniform.
banging your head on the desk, you screamed internally.
what did i do to deserve this? whatever, i can ignore him during class.
unfortunately for you, however, you could not ignore him during class. because the teacher had decided to sit him next to you.
"wait, how do i do this question?" he whispered.
"i don't know," you lied, "i wasn't even listening just now."
frowning at you, he gave up. he had reluctantly been making contact with you to grasp the lesson more, but you weren't of any use.
soon enough, it was lunch time.
joyfully, you grabbed your food and made your way to the lunch hall. finally, you would get the much-desired alone-time you had deserved. sitting with h/n was tiring, due to his constant questions. you hated hearing his voice even briefly in the pc room, it was unbearable at school. how were you going to last with him sitting next to you for the rest of the school year?
running a hand through your hair, you opened your lunch to find a sandwich and a container of cherries.
you plopped one of the cherries in your mouth, savouring the explosion of flavour on your tastebuds.
you noticed how when you picked one up, it had left its red juice on your finger, as a reminder of the warm feeling you had when eating it.
just as you were about to bite into another, you heard a deep thud.
looking up, you were met with the annoyingly chocolatey-brown eyes of h/n.
"why are you here?" you asked, ready to snap at him.
"i have literally no friends here, where else would i sit?"
"you had like 10 million girls throwing heart eyes at you when you arrived, i'm sure they'd enjoy your presence."
he was genuinely taken aback, "i did?"
"yeah, you did. do you not have the capacity to interpret social queues?"
"no i just," he whispered, embarrassed, "never really had that many people like me that way before."
"likewise," you snorted, "now, let me eat my lunch in peace."
and he did as you asked.
the two of you sat, in dead silence, eating your food. anything you may have said would have been drowned out by the loudness of the rest of the students anyways, so, it was best not to make conversation.
after concluding your meal, you wordlessly left him and went to the bathroom.
"where are you going?" he stopped you.
"the jungle."
and you left.
your trip to the bathroom was as usual. until it was not.
two girls came up to you, hands on their hips. it was quite amusing, actually, they looked like monkeys.
"what's with you and the new kid?" one of them questioned.
you started at them blankly. were they serious? this wasn't some kdrama, they couldn't stop you from spending time with him.
well, it's not like you wanted to anyways.
you tried to leave, but they stopped you.
"you sit alone until today, but when h/n comes you act all nice?"
"listen here you bit-"
you were about to teach them a lesson when you heard a familiar voice calling out your name.
"y/n- oh hey." he greeted, sheepishly, "i need your help, where's the biology room? i don't know my way around school yet-"
"i'll show you." you replied, begrudgingly.
great, now these stupid girls thought you and h/n had something going on. how could they have it so backwards?
you grabbed his arm roughly, already fuming in anger as you strutted down the halls, leading him to the biology classroom.
"there? you happy now?"
"um- yeah?" he tilted his head in confusion, "so uh, why are you so angry all of a sudden?"
"i don't want to talk about it." you grit you teeth.
"alright." h/n squeaked.
the rest of the day was fine, h/n didn't try to talk to you once, which was nice. not having to deal with him certainly brought down your stress levels. maybe you had scared him off with the anger.
you sat in class, extremely bored, waiting for the bell to ring so that you could finally evacuate this stupid school building and head to the pc room, your current safe haven.
sure, h/n was there, but at least, if you put on headphones, you could ignore him.
and there it was, the glorious sound of the bell.
it was like the sound of victory when you won a race in kart rider, and it was like the sensation when you had finally defeated that one annoying opponent in overwatch.
it felt like freedom. and frankly, that's exactly what it was.
getting up from your seat excitedly, you speed-walked in the direction of the pc room, longing to play the games you had yearned for during the course of the tiring school day.
hearing something shuffle behind you, you tensed.
was something there? you couldn't help but wonder.
slyly pretending to have dropped something, you took a glance behind you as you bent to pick it up, finding only h/n.
wait, h/n?
"why are you following me?" you interrogated.
"following you? i think you've forgotten we share the exact same evening routine."
oh, right.
sighing in defeat, you walked beside him this time, hoping that if there actually was anyone following you, he would be kind enough to help you if there was a crisis.
and what seemed like hours of walking, you had finally arrived.
rushing to your spot, you turned on the pc, wanting to escape from this reality as fast as possible. but unfortunately, h/n was apparently determined to ruin your day.
"play games with me."
you couldn't believe your ears.
"y-you want me to what?" you sat back in shock.
"you heard me."
you shook your head in disbelief, "look here, just because i put up with you for a day doesn't mean we're friends or anything-"
"no, it's not that, you narcissist, i'm bored, play games with me."
"no."
"y/n, please." he begged.
you looked into his eyes. you didn't know why, but he looked strangely cute? no. that couldn't be. you found him repulsive, right?
"fine," you begrudgingly agreed, "just this one time."
he smiled in delight and dragged you over to his pc.
you joined the game, and it was surprisingly more fun than you had thought.
you heard him laugh for the first time, which was new, but strange. strange in a good way, somehow.
"you're not so bad when you're not being a dickhead." you admitted.
"shut the fuck up and play your game."
"how can i when you use cherry mx blues?"
he turned in his chair.
"are you insulting my switches?" he choked, mouth agape.
"yes," you acknowledged, "mx blues are trash, use a good switch like a linear? have you ever heard of nk creams? gateron ink blacks? mx browns even??"
he looked as if you had just hurt him in the worst way possible, which, you really had.
"i'm not talking to you anymore."
you laughed at him warmly. the more time you spent with him, the nicer he seemed to get. and if that was true, you'd be willing to spend all the time in the world with him.
92 notes · View notes
angrelysimpping · 3 years ago
Note
could you write bailey somehow getting trapped inside the loft (the orphanage being empty aside from them, maybe there is some sort of carnival in town?) and pc rescuing and then helping bailey calm down?...i am a fan of hurt/comfort
I kept rewriting this cause I kept feeling like I was making Bailey too ooc but they're dealing with past trauma so fuck it, they can be ooc =P
(Uh, angst; implied past physical/sexual abuse; slight dissociation [?])
A carnival was in town. The brats had all seemed excited. When Bailey gave their blessing, had let them all go out that day, had even given them some money for games, they were surprised.
You were surprised, giving Bailey a questioning look as you helped Robin round everyone up. But you still left with them, leaving Bailey alone.
They waited until the orphanage was empty to go up. Waited 20 minutes, just in case a brat came stumbling back in, having forgotten something. They wanted to make sure they were alone for this.
Had thought, if they just faced their fears, it would help.
It might have, if the ladder hadn't folded up behind them, the latch banging shut.
The panic sets in immediately, the sound dredging up long-suppressed memories.
They used to have nightmares about that sound, back when they still dreamed.
Bailey grapples with the latch, but it’s hard to open from the inside. Had been designed that way.
It doesn't help that Bailey's hands are starting to shake.
Bailey reels back, cursing.
They had managed to cut their palm open on the latch.
They've never had a problem with blood before. They've made others bleed, they've been made to bleed. It's common in their work.
Maybe it's the limited lighting in the loft. The warmth or the stale air.
Whatever the reason, Bailey feels light-headed as they watch the blood well up in their hand.
Detached, unreal.
It’s hard for Bailey to get to their feet. Body heavy, head full of cotton.
They end up in a corner, staring at the hatch.
They’re not sure, but they think they've been here before, in this corner. Oh, yes. Yes, they have. Look, there are the scratch marks. They remember making those. If they listen hard enough, Bailey is sure they'll be able to hear crying a room over.
Bailey never cried. Not before, not now.
What a time to leave their gun downstairs, huh?
Then again, would it even help? What were they going to do, shoot the latch open?
Did it even matter?
They’re doing their best to think, fighting the panic that thrums through their veins, darkens the edges of their vision. It would be so easy, to let themself be pulled into unconsciousness.
But they refuse.
They’re vaguely aware that they’ve started scratching at their neck, an old habit. One they thought they’d outgrown.
The feeling muted, barely there.
They're aware that it means something, something bad, that they’re losing feeling. That they’re going numb.
They should get up.
Anything. They should do anything. Anything that wasn’t just sitting here.
They try to stand, but their body doesn't move like they want it to. They sit again, letting their head thump back against the wall.
The sound is muffled by the ringing in their ears, but they can hear it, even if they don't feel it properly. That's good. They think. They're not sure.
Someone is calling their name.
It's a familiar voice, but they can't quite place it, mind hazy. They try to call back, but the sound gets caught in their throat. All that comes out is a hoarse groan.
They can’t process that it's you at first when you appear next to them. It’s when you start to say their name, pull on their arm, that they recognize you.
Bailey doesn't flinch when you touch them, so that’s good, they think. That they can actually feel you touching them, lifting them to their feet, guiding them towards the hatch.
Climbing down the ladder takes a monumental amount of effort. They have to go slow, struggling to grip the ladder properly.
The moment Bailey exits the loft, they feel a little clearer, a little more solid.
And utterly exhausted.
Now free from the loft, the panic subsiding, they feel hollow.
They want to sleep. They want to sleep for a thousand years and then some. And to never go into the loft again.
But they still have work they need to do.
Fuck, it’s going to be a long day.
Bailey starts towards their office, but you grab their arm, pulling them into the nearest bathroom.
"Fuck off," Bailey mumbles, voice thick.
"Let me clean you up."
They grumble but allow it. Might as well. Maybe it will help in the long run.
You try to get them to sit on the toilet, but they refuse. If they sit now, they're not sure they'd be able to get back up.
Giving up, you grab a washcloth and start to dab at their face.
They’d gotten blood on their face, at some point, they’re not sure when.
You're muttering to yourself while looking for the source.
They’re about to show you the gash on their hand, let you know where the blood came from. Maybe then they can get to work, put this all behind them.
But then you start to comb through their hair.
“Head wounds would cause more blood, but maybe if it was small-?”
You’re interrupted as Bailey leans into your touch, sighing.
When was the last time someone had touched them like this? With so much care? Bailey can’t remember.
"B-Bailey?"
They don't respond, wrapping an arm around you and holding you against them.
Bailey buries their face into the crook of your neck, inhaling deeply. God, you’re so warm. And solid.
Fuck, if they don’t sit soon they’re going to collapse and you’ll have to carry them. They couldn’t live that down, even if you let them.
You're about to say something when Bailey pulls away, starts to move as if to leave the bathroom.
"You're bleeding. Your face-"
Bailey holds out their hand, showing you the cut.
They can see the relief wash over you as you realize there wasn't a hidden head wound to worry about on top of everything else.
They push past you, headed to their room.
They only pause in their office for a second before going into their bedroom.
They’re too fucking drained to work.
Bailey flops face down on their bed.
With their uninjured hand, they check the gun under their pillow. The cool metal is the only reassurance they need to allow themself to relax.
They don’t realize they’ve dozed off until you wake them, bandaging their hand.
Didn't they lock the door? Maybe. They can't really remember.
You’re muttering about how it doesn't look bad enough to need stitches, how they might still need to see a doctor.
You finish bandaging their hand and pause.
They expect you to leave.
But you don’t.
Bailey can hear you moving around their room, locking the door, and checking the window.
They’re starting to drift off again when you approach the bed.
Bailey reaches out, grabbing your forearm. They mumble something incomprehensible even to their own ears, tugging you into the bed with them.
You don't fight Bailey on it, letting them press you against them.
They fall into a deep sleep with an arm wrapped around you and their face pressed into your neck.
It's the best sleep they've had in a long time.
When they wake up, you're still in their bed. Sill pressed against them.
Their movement wakes you, and Bailey watches as a bush burns across your face at being in such an intimate position with them.
"If you know what's good for you, you won't tell a soul about this. Got it?"
You nod your head.
Bailey sighs before rolling out of bed.
They ignore you as they change clothes.
They're still tired, but they can't afford a day off. They wouldn't want one, either. After pissing the whole day away yesterday, they refuse to waste another.
When you flee their room, Bailey thinks it's all over.
They're wrong.
You show back up in their office minutes later in fresh clothes, holding a book.
Bailey stares at you as you make yourself comfortable in the corner of their office, but they don't chase you off.
Bailey knows what you're doing. Knows you're trying to keep an eye on them. They won't stop you.
They won't snap at you to get out of their office. You’re just sitting in the corner and reading, anyway.
Bailey actually forgets you’re in the room with them at one point. Jumping when you speak up, reminding them to eat.
They don't say anything as it gets closer to midnight and you still haven’t left their side.
Or when you follow them back into their bedroom.
112 notes · View notes
kero-verdade · 2 years ago
Note
ohhh about cyber security, do you have any tips? something recently happened that someone hacked my freaking soundcloud that i didnt even remember existed. it was like 10 yrs old.. and i was so freaked out someone hacked it. i just ended up deleting it. i still have the same email and im not sure if i should make a new one. i honestly dont remember what my pw was but my email was in a few data breaches (myfitnesspal, some math app, chegg) like i had really bad luck but i just changed my pw and moved on but idk if i should be doing more? things like that creep me out cus im very private but i also dont use a password manager cus somehow i feel like i would be even more vulnerable :/ its so weird the idea of giving an application the job to store passwords for you idk if u could educate me
For sure! Big ol rant under the cut. And if you have any questions or if I missed anything, pls let me know :D
With your email, you don't necessarily need to make a new one! Depending on what email domain you're using, you have lots of security options.
For starters, definitely change your email password whenever you're notified that it's been in a breach. And in general, it's a good practice to not repeat the same password between different websites - that way if one of your accounts ever gets hacked, the same hacker can't use that username/password to hack into all your other accounts on other websites too. TBH this is something I'm bad about because having the same password across websites is so much easier to remember, but ultimately it's the safer thing to do!
Most websites use single sign-on now where you only need your existing Google/Facebook/etc account to log in, so in that case having different passwords across websites wouldn't apply. But if you do remember any websites where you needed to create a new account, try resetting their passwords just to see if you can! I recently dug up my 14-year-old Roblox account for just this reason lmao. And if you have time, also try to delete accounts on websites that you're no longer using. But I wouldn't stress over it too much, since that's a lot of ground to cover. I don't know if SoundCloud notified you of the hack or if you found out on your own, but the important part is you were made aware of it and were able to deal with it.
Next, check if you can turn on two-factor authentication for your email account (and any other important account - bank accounts, etc). The most common kind is when you get a code sent to your phone that you have to verify before you log in, but there's lots of different kinds. That way, even if someone gets hold of your email credentials, they won't necessarily be able to log in. Gmail in particular is VERY good about this, and Outlook has good options too.
This is optional, but if you want to take an EXTRA security step, you could even try having multiple emails. For me, I have a primary email that I use for work/school/professional stuff, and another old email that I only use to sign up for websites/mailing lists/spammy stuff. So even if my second email got hacked or leaked, it would suck a LOT, but it wouldn't be the end of the world since all of my important stuff is in a different place. But ofc checking multiple inboxes can get annoying too, so it's really whatever you prefer.
As for password managers, that's really your own preference as well. I personally don't use one because I prefer to write down my passwords in a safe place, and that works out fine for me. I also have friends who swear by password managers, reputable password managers are very secure if you use them properly. I personally DON'T recommend using password managers on phones/tablets/any device that can be easily lost or stolen, but they can definitely come in handy on a PC or similar. There's no wrong answer, so just do whatever makes you more comfortable!
If you read all of that congratulations, I hope this was at least a bit helpful!
And since we're on the topic, one more thing I gotta ramble about rq:
Be careful with your biometric data! Things like FaceID and fingerprint are SUPER convenient for unlocking your devices without needing to remember a bunch of passwords, but the laws around them are still developing. Depending on what country/state you're in, existing privacy laws usually haven't caught up with technology, and that information about you may not have the same legal protections as regular passwords. This really only matters if your phone gets seized by police or something super crazy like that, but I think it's good to be aware of since biometric logins are becoming so much more common.
4 notes · View notes
itbeatsbookmarks · 4 years ago
Link
(Via: Hacker News)
The log/event processing pipeline you can't have
Let me tell you about the still-not-defunct real-time log processing pipeline we built at my now-defunct last job. It handled logs from a large number of embedded devices that our ISP operated on behalf of residential customers. (I wrote and presented previously about some of the cool wifi diagnostics that were possible with this data set.)
Lately, I've had a surprisingly large number of conversations about logs processing pipelines. I can find probably 10+ already-funded, seemingly successful startups processing logs, and the Big Name Cloud providers all have some kind of logs thingy, but still, people are not satisfied. It's expensive and slow. And if you complain, you mostly get told that you shouldn't be using unstructured logs anyway, you should be using event streams.
That advice is not wrong, but it's incomplete.
Instead of doing a survey of the whole unhappy landscape, let's just ignore what other people suffer with and talk about what does work. You can probably find, somewhere, something similar to each of the components I'm going to talk about, but you probably can't find a single solution that combines it all with good performance and super-low latency for a reasonable price. At least, I haven't found it. I was a little surprised by this, because I didn't think we were doing anything all that innovative. Apparently I was incorrect.
The big picture
Let's get started. Here's a handy diagram of all the parts we're going to talk about:
The ISP where I worked has a bunch of embedded Linux devices (routers, firewalls, wifi access points, and so on) that we wanted to monitor. The number increased rapidly over time, but let's talk about a nice round number, like 100,000 of them. Initially there were zero, then maybe 10 in our development lab, and eventually we hit 100,000, and later there were many more than that. Whatever. Let's work with 100,000. But keep in mind that this architecture works pretty much the same with any number of devices.
(It's a "distributed system" in the sense of scalability, but it's also the simplest thing that really works for any number of devices more than a handful, which makes it different from many "distributed systems" where you could have solved the problem much more simply if you didn't care about scaling. Since our logs are coming from multiple sources, we can't make it non-distributed, but we can try to minimize the number of parts that have to deal with the extra complexity.)
Now, these are devices we were monitoring, not apps or services or containers or whatever. That means two things: we had to deal with lots of weird problems (like compiler/kernel bugs and hardware failures), and most of the software was off-the-shelf OS stuff we couldn't easily control (or didn't want to rewrite).
(Here's the good news: because embedded devices have all the problems from top to bottom, any solution that works for my masses of embedded devices will work for any other log-pipeline problem you might have. If you're lucky, you can leave out some parts.)
That means the debate about "events" vs "logs" was kind of moot. We didn't control all the parts in our system, so telling us to forget logs and use only structured events doesn't help. udhcpd produces messages the way it wants to produce messages, and that's life. Sometimes the kernel panics and prints whatever it wants to print, and that's life. Move on.
Of course, we also had our own apps, which means we could also produce our own structured events when it was relevant to our own apps. Our team had whole never-ending debates about which is better, logs or events, structured or unstructured. In fact, in a move only overfunded megacorporations can afford, we actually implemented both and ran them both for a long time.
Thus, I can now tell you the final true answer, once and for all: you want structured events in your database.
...but you need to be able to produce them from unstructured logs. And once you can do that, exactly how those structured events are produced (either from logs or directly from structured trace output) turns out to be unimportant.
But we're getting ahead of ourselves a bit. Let's take our flow diagram, one part at a time, from left to right.
Userspace and kernel messages, in a single stream
Some people who have been hacking on Linux for a while may know about /proc/kmsg: that's the file good old (pre-systemd) klogd reads kernel messages from, and pumps them to syslogd, which saves them to a file. Nowadays systemd does roughly the same thing but with more d-bus and more corrupted binary log files. Ahem. Anyway. When you run the dmesg command, it reads the same kernel messages (in a slightly different way).
What you might not know is that you can go the other direction. There's a file called /dev/kmsg (note: /dev and not /proc) which, if you write to it, produces messages into the kernel's buffer. Let's do that! For all our messages!
Wait, what? Am I crazy? Why do that?
Because we want strict sequencing of log messages between programs. And we want that even if your kernel panics.
Imagine you have, say, a TV DVR running on an embedded Linux system, and whenever you go to play a particular recorded video, the kernel panics because your chipset vendor hates you. Hypothetically. (The feeling is, hypothetically, mutual.) Ideally, you would like your logs to contain a note that the user requested the video, the video is about to start playing, we've opened the file, we're about to start streaming the file to the proprietary and very buggy (hypothetical) video decoder... boom. Panic.
What now? Well, if you're writing the log messages to disk, the joke's on you, because I bet you didn't fsync() after each one. (Once upon a time, syslogd actually did fsync() after each one. It was insanely disk-grindy and had very low throughput. Those days are gone.) Moreover, a kernel panic kills the disk driver, so you have no chance to fsync() it after the panic, unless you engage one of the more terrifying hacks like, after a panic, booting into a secondary kernel whose only job is to stream the message buffer into a file, hoping desperately that the disk driver isn't the thing that panicked, that the disk itself hasn't fried, and that even if you do manage to write to some disk blocks, they are the right ones because your filesystem data structure is reasonably intact.
(I suddenly feel a lot of pity for myself after reading that paragraph. I think I am more scars than person at this point.)
ANYWAY
The kernel log buffer is in a fixed-size memory buffer in RAM. It defaults to being kinda small (tens or hundreds of kBytes), but you can make it bigger if you want. I suggest you do so.
By itself, this won't solve your kernel panic problems, because RAM is even more volatile than disk, and you have to reboot after a kernel panic. So the RAM is gone, right?
Well, no. Sort of. Not exactly.
Once upon a time, your PC BIOS would go through all your RAM at boot time and run a memory test. I remember my ancient 386DX PC used to do this with my amazingly robust and life-changing 4MB of RAM. It took quite a while. You could press ESC to skip it if you were a valiant risk-taking rebel like myself.
Now, memory is a lot faster than it used to be, but unfortunately it has gotten bigger more quickly than it has gotten faster, especially if you disable memory caching, which you certainly must do at boot time in order to write the very specific patterns needed to see if there are any bit errors.
So... we don't do the boot-time memory test. That ended years ago. If you reboot your system, the memory mostly will contain the stuff it contained before you rebooted. The OS kernel has to know that and zero out pages as they get used. (Sometimes the kernel gets fancy and pre-zeroes some extra pages when it's not busy, so it can hand out zero pages more quickly on demand. But it always has to zero them.)
So, the pages are still around when the system reboots. What we want to happen is:
The system reboots automatically after a kernel panic. You can do this by giving your kernel a boot parameter like "panic=1", which reboots it after one second. (This is not nearly enough time for an end user to read and contemplate the panic message. That's fine, because a) on a desktop PC, X11 will have crashed in graphics mode so you can't see the panic message anyway, and b) on an embedded system there is usually no display to put the message on. End users don't care about panic messages. Our job is to reboot, ASAP, so they don't try to "help" by power cycling the device, which really does lose your memory.) (Advanced users will make it reboot after zero seconds. I think panic=0 disables the reboot feature rather than doing that, so you might have to patch the kernel. I forget. We did it, whatever it was.)
The kernel always initializes the dmesg buffer in the same spot in RAM.
The kernel notices that a previous dmesg buffer is already in that spot in RAM (because of a valid signature or checksum or whatever) and decides to append to that buffer instead of starting fresh.
In userspace, we pick up log processing where we left off. We can capture the log messages starting before (and therefore including) the panic!
And because we redirected userspace logs to the kernel message buffer, we have also preserved the exact sequence of events that led up to the panic.
If you want all this to happen, I have good news and bad news. The good news is we open sourced all our code; the bad news is it didn't get upstreamed anywhere so there are no batteries included and no documentation and it probably doesn't quite work for your use case. Sorry.
Open source code:
logos tool for sending userspace logs to /dev/klogd. (It's logs... for the OS.. and it's logical... and it brings your logs back from the dead after a reboot... get it? No? Oh well.) This includes two per-app token buckets (burst and long-term) so that an out-of-control app won't overfill the limited amount of dmesg space.
PRINTK_PERSIST patch to make Linux reuse the dmesg buffer across reboots.
Even if you don't do any of the rest of this, everybody should use PRINTK_PERSIST on every computer, virtual or physical. Seriously. It's so good.
(Note: room for improvement: it would be better if we could just redirect app stdout/stderr directly to /dev/kmsg, but that doesn't work as well as we want. First, it doesn't auto-prefix incoming messages with the app name. Second, libc functions like printf() actually write a few bytes at a time, not one message per write() call, so they would end up producing more than one dmesg entry per line. Third, /dev/kmsg doesn't support the token bucket rate control that logos does, which turns out to be essential, because sometimes apps go crazy. So we'd have to further extend the kernel API to make it work. It would be worthwhile, though, because the extra userspace process causes an unavoidable delay between when a userspace program prints something and when it actually gets into the kernel log. That delay is enough time for a kernel to panic, and the userspace message gets lost. Writing directly to /dev/kmsg would take less CPU, leave userspace latency unchanged, and ensure the message is safely written before continuing. Someday!)
(In related news, this makes all of syslogd kinda extraneous. Similarly for whatever systemd does. Why do we make everything so complicated? Just write directly to files or the kernel log buffer. It's cheap and easy.)
Uploading the logs
Next, we need to get the messages out of the kernel log buffer and into our log processing server, wherever that might be.
(Note: if we do the above trick - writing userspace messages to the kernel buffer - then we can't also use klogd to read them back into syslogd. That would create an infinite loop, and would end badly. Ask me how I know.)
So, no klogd -> syslogd -> file. Instead, we have something like syslogd -> kmsg -> uploader or app -> kmsg -> uploader.
What is a log uploader? Well, it's a thing that reads messages from the kernel kmsg buffer as they arrive, and uploads them to a server, perhaps over https. It might be almost as simple as "dmesg | curl", like my original prototype, but we can get a bit fancier:
Figure out which messages we've already uploaded (eg. from the persistent buffer before we rebooted) and don't upload those again.
Log the current wall-clock time before uploading, giving us sync points between monotonic time (/dev/kmsg logs "microseconds since boot" by default, which is very useful, but we also want to be able to correlate that with "real" time so we can match messages between related machines).
Compress the file on the way out.
Somehow authenticate with the log server.
Bonus: if the log server is unavailable because of a network partition, try to keep around the last few messages from before the partition, as well as the recent messages once the partition is restored. If the network partition was caused by the client - not too rare if you, like us, were in the business of making routers and wifi access points - you really would like to see the messages from right before the connectivity loss.
Luckily for you, we also open sourced our code for this. It's in C so it's very small and low-overhead. We never quite got the code for the "bonus" feature working quite right, though; we kinda got interrupted at the last minute.
Open source code:
loguploader C client, including an rsyslog plugin for Debian in case you don't want to use the /dev/kmsg trick.
devcert, a tool (and Debian package) which auto-generates a self signed "device certificate" wherever it's installed. The device certificate is used by a device (or VM, container, whatever) to identify itself to the log server, which can then decide how to classify and store (or reject) its logs.
One thing we unfortunately didn't get around to doing was modifying the logupload client to stream logs to the server. This is possible using HTTP POST and Chunked encoding, but our server at the time was unable to accept streaming POST requests due to (I think now fixed) infrastructure limitations.
(Note: if you write load balancing proxy servers or HTTP server frameworks, make sure they can start processing a POST request as soon as all the headers have arrived, rather than waiting for the entire blob to be complete! Then a log upload server can just stream the bytes straight to the next stage even before the whole request has finished.)
Because we lacked streaming in the client, we had to upload chunks of log periodically, which leads to a tradeoff about what makes a good upload period. We eventually settled on about 60 seconds, which ended up accounting for almost all the end-to-end latency from message generation to our monitoring console.
Most people probably think 60 seconds is not too bad. But some of the awesome people on our team managed to squeeze all the other pipeline phases down to tens of milliseconds in total. So the remaining 60 seconds (technically: anywhere from 0 to 60 seconds after a message was produced) was kinda embarrassing. Streaming live from device to server would be better.
The log receiver
So okay, we're uploading the logs from client to some kind of server. What does the server do?
This part is both the easiest and the most reliability-critical. The job is this: receive an HTTP POST request, write the POST data to a file, and return HTTP 200 OK. Anybody who has any server-side experience at all can write this in their preferred language in about 10 minutes.
We intentionally want to make this phase as absolutely simplistic as possible. This is the phase that accepts logs from the limited-size kmsg buffer on the client and puts them somewhere persistent. It's nice to have real-time alerts, but if I have to choose between somewhat delayed alerts or randomly losing log messages when things get ugly, I'll have to accept the delayed alerts. Don't lose log messages! You'll regret it.
The best way to not lose messages is to minimize the work done by your log receiver. So we did. It receives the uploaded log file chunk and appends it to a file, and that's it. The "file" is actually in a cloud storage system that's more-or-less like S3. When I explained this to someone, they asked why we didn't put it in a Bigtable-like thing or some other database, because isn't a filesystem kinda cheesy? No, it's not cheesy, it's simple. Simple things don't break. Our friends on the "let's use structured events to make metrics" team streamed those events straight into a database, and it broke all the time, because databases have configuration options and you inevitably set those options wrong, and it'll fall over under heavy load, and you won't find out until you're right in the middle of an emergency and you really want to see those logs. Or events.
Of course, the file storage service we used was encrypted-at-rest, heavily audited, and auto-deleted files after N days. When you're a megacorporation, you have whole teams of people dedicated to making sure you don't screw this up. They will find you. Best not to annoy them.
We had to add one extra feature, which was authentication. It's not okay for random people on the Internet to be able to impersonate your devices and spam your logs - at least without putting some work into it. For device authentication, we used the rarely-used HTTP client-side certificates option and the devcert program (linked above) so that the client and server could mutually authenticate each other. The server didn't check the certificates against a certification authority (CA), like web clients usually do; instead, it had a database with a whitelist of exactly which certs we're allowing today. So in case someone stole a device cert and started screwing around, we could remove their cert from the whitelist and not worry about CRL bugs and latencies and whatnot.
Unfortunately, because our log receiver was an internal app relying on internal infrastructure, it wasn't open sourced. But there really wasn't much there, honest. The first one was written in maybe 150 lines of python, and the replacement was rewritten in slightly more lines of Go. No problem.
Retries and floods
Of course, things don't always go smoothly. If you're an ISP, the least easy thing is dealing with cases where a whole neighbourhood gets disconnected, either because of a power loss or because someone cut the fiber Internet feed to the neighbourhood.
Now, disconnections are not such a big deal for logs processing - you don't have any. But reconnection is a really big deal. Now you have tens or hundreds of thousands of your devices coming back online at once, and a) they have accumulated a lot more log messages than they usually do, since they couldn't upload them, and b) they all want to talk to your server at the same time. Uh oh.
Luckily, our system was designed carefully (uh... eventually it was), so it could handle these situations pretty smoothly:
The log uploader uses a backoff timer so that if it's been trying to upload for a while, it uploads less often. (However, the backoff timer was limited to no more than the usual inter-upload interval. I don't know why more people don't do this. It's rather silly for your system to wait longer between uploads in a failure situation than it would in a success situation. This is especially true with logs, where when things come back online, you want a status update now. And clearly your servers have enough capacity to handle uploads at the usual rate, because they usually don't crash. Sorry if I sound defensive here, but I had to have this argument a few times with a few SREs. I understand why limiting the backoff period isn't always the right move. It's the right move here.)
Less obviously, even under normal conditions, the log uploader uses a randomized interval between uploads. This avoids traffic spikes where, after the Internet comes back online, everybody uploads again exactly 60 seconds later, and so on.
The log upload client understands the idea that the server can't accept its request right now. It has to, anyway, because if the Internet goes down, there's no server. So it treats server errors exactly like it treats lack of connectivity. And luckily, log uploading is not really an "interactive" priority task, so it's okay to sacrifice latency when things get bad. Users won't notice. And apparently our network is down, so the admins already noticed.
The /dev/kmsg buffer was configured for the longest reasonable outage we could expect, so that it wouldn't overflow during "typical" downtime. Of course, there's a judgement call here. But the truth is, if you're having system-wide downtime, what the individual devices were doing during that downtime is not usually what you care about. So you only need to handle, say, the 90th percentile of downtime. Safely ignore the black swans for once.
The log receiver aggressively rejects requests that come faster than its ability to write files to disk. Since the clients know how to retry with a delay, this allows us to smooth out bursty traffic without needing to either over-provision the servers or lose log messages.
(Pro tip, learned the hard way: if you're writing a log receiver in Go, don't do the obvious thing and fire off a goroutine for every incoming request. You'll run out of memory. Define a maximum number of threads you're willing to handle at once, and limit your request handling to that. It's okay to set this value low, just to be safe: remember, the uploader clients will come back later.)
Okay! Now our (unstructured) logs from all our 100,000 devices are sitting safely in a big distributed filesystem. We have a little load-balanced, multi-homed cluster of log receivers accepting the uploads, and they're so simple that they should pretty much never die, and even if they do because we did something dumb (treacherous, treacherous goroutines!), the clients will try again.
What might not be obvious is this: our reliability, persistence, and scaling problems are solved. Or rather, as long as we have enough log receiver instances to handle all our devices, and enough disk quota to store all our logs, we will never again lose a log message.
That means the rest of our pipeline can be best-effort, complicated, and frequently exploding. And that's a good thing, because we're going to start using more off-the-shelf stuff, we're going to let random developers reconfigure the filtering rules, and we're not going to bother to configure it with any redundancy.
Grinding the logs
The next step is to take our unstructured logs and try to understand them. In other words, we want to add some structure. Basically we want to look for lines that are "interesting" and parse out the "interesting" data and produce a stream of events, each with a set of labels describing what categories they apply to.
Note that, other than this phase, there is little difference between how you'd design a structured event reporting pipeline and a log pipeline. You still need to collect the events. You still (if you're like me) need to persist your events across kernel panics. You still need to retry uploading them if your network gets partitioned. You still need the receivers to handle overloading, burstiness, and retries. You still would like to stream them (if your infrastructure can handle it) rather than uploading every 60 seconds. You still want to be able to handle a high volume of them. You're just uploading a structured blob instead of an unstructured blob.
Okay. Fine. If you want to upload structured blobs, go for it. It's just an HTTP POST that appends to a file. Nobody's stopping you. Just please try to follow my advice when designing the parts of the pipeline before and after this phase, because otherwise I guarantee you'll be sad eventually.
Anyway, if you're staying with me, now we have to parse our unstructured logs. What's really cool - what makes this a killer design compared to starting with structured events in the first place - is that we can, at any time, change our minds about how to parse the logs, without redeploying all the software that produces them.
This turns out to be amazingly handy. It's so amazingly handy that nobody believes me. Even I didn't believe me until I experienced it; I was sure, in the beginning, that the unstructured logs were only temporary and we'd initially use them to figure out what structured events we wanted to record, and then modify the software to send those, then phase out the logs over time. This never happened. We never settled down. Every week, or at least every month, there was some new problem which the existing "structured" events weren't configured to catch, but which, upon investigating, we realized we could diagnose and measure from the existing log message stream. And so we did!
Now, I have to put this in perspective. Someone probably told you that log messages are too slow, or too big, or too hard to read, or too hard to use, or you should use them while debugging and then delete them. All those people were living in the past and they didn't have a fancy log pipeline. Computers are really, really fast now. Storage is really, really cheap.
So we let it all out. Our devices produced an average of 50 MB of (uncompressed) logs per day, each. For the baseline 100,000 devices that we discussed above, that's about 5TB of logs per day. Ignoring compression, how much does it cost to store, say, 60 days of logs in S3 at 5TB per day? "Who cares," that's how much. You're amortizing it over 100,000 devices. Heck, a lot of those devices were DVRs, each with 2TB of storage. With 100,000 DVRs, that's 200,000 TB of storage. Another 300 is literally a rounding error (like, smaller than if I can't remember if it's really 2TB or 2TiB or what).
Our systems barfed up logs vigorously and continuously, like a non-drunken non-sailor with seasickness. And it was beautiful.
(By the way, now would be a good time to mention some things we didn't log: personally identifiable information or information about people's Internet usage habits. These were diagnostic logs for running the network and detecting hardware/software failures. We didn't track what you did with the network. That was an intentional decision from day 1.)
(Also, this is why I think all those log processing services are so badly overpriced. I wanna store 50 MB per device, for lots of devices. I need to pay S3 rates for that, not a million dollars a gigabyte. If I have to overpay for storage, I'll have to start writing fewer logs. I love my logs. I need my logs. I know you're just storing it in S3 anyway. You probably get a volume discount! Let's be realistic.)
But the grinding, though
Oh right. So the big box labeled "Grinder" in my diagram was, in fact, just one single virtual machine, for a long time. It lasted like that for much longer than we expected.
Whoa, how is that possible, you ask?
Well, at 5TB per day per 100,000 devices, that's an average of 57 MBytes per second. And remember, burstiness has already been absorbed by our carefully written log receivers and clients, so we'll just grind these logs as fast as they arrive or as fast as we can, and if there are fluctuations, they'll average out. Admittedly, some parts of the day are busier than others. Let's say 80 MBytes per second at peak.
80 MBytes per second? My laptop can do that on its spinning disk. I don't even need an SSD! 80 MBytes per second is a toy.
And of course, it's not just one spinning disk. The data itself is stored on some fancy heavily-engineered distributed filesystem that I didn't have to design. Assuming there are no, er, collossal, failures in provisioning (no comment), there's no reason we shouldn't be able to read files at a rate that saturates the network interface available to our machine. Surely that's at least 10 Gbps (~1 GByte/sec) nowadays, which is 12.5 of those. 1.25 million devices, all processed by a single grinder.
Of course you'll probably need to use a few CPU cores. And the more work you do per log entry, the slower it'll get. But these estimates aren't too far off what we could handle.
And yeah, sometimes that VM gets randomly killed by the cluster's Star Trek-esque hive mind for no reason. It doesn't matter, because the input data was already persisted by the log receivers. Just start a new grinder and pick up where you left off. You'll have to be able to handle process restarts no matter what. And that's a lot easier than trying to make a distributed system you didn't need.
As for what the grinder actually does? Anything you want. But it's basically the "map" phase in a mapreduce. It reads the data in one side, does some stuff to it, and writes out postprocessed stuff on the other side. Use your imagination. And if you want to write more kinds of mappers, you can run them, either alongside the original Grinder or downstream from it.
Our Grinder mostly just ran regexes and put out structures (technically protobufs) that were basically sets of key-value pairs.
(For some reason, when I search the Internet for "streaming mapreduce," I don't get programs that do this real-time processing of lots of files as they get written. Instead, I seem to get batch-oriented mapreduce clones that happen to read from stdin, which is a stream. I guess. But... well, now you've wasted some perfectly good words that could have meant something. So okay, too bad, it's a Grinder. Sue me.)
Reducers and Indexers
Once you have a bunch of structured events... well, I'm not going to explain that in a lot of detail, because it's been written about a lot.
You probably want to aggregate them a bit - eg. to count up reboots across multiple devices, rather than storing each event for each device separately - and dump them into a time-series database. Perhaps you want to save and postprocess the results in a monitoring system named after Queen Elizabeth or her pet butterfly. Whatever. Plug in your favourite.
What you probably think you want to do, but it turns out you rarely need, is full-text indexing. People just don't grep the logs across 100,000 devices all that often. I mean, it's kinda nice to have. But it doesn't have to be instantaneous. You can plug in your favourite full text indexer if you like. But most of the time, just an occasional big parallel grep (perhaps using your favourite mapreduce clone or something more modern... or possibly just using grep) of a subset of the logs is sufficient.
(If you don't have too many devices, even a serial grep can be fine. Remember, a decent cloud computer should be able to read through ~1 GByte/sec, no problem. How much are you paying for someone to run some bloaty full-text indexer on all your logs, to save a few milliseconds per grep?)
I mean, run a full text indexer if you want. The files are right there. Don't let me stop you.
On the other hand, being able to retrieve the exact series of logs - let's call it the "narrative" - from a particular time period across a subset of devices turns out to be super useful. A mini-indexer that just remembers which logs from which devices ended up in which files at which offsets is nice to have. Someone else on our team built one of those eventually (once we grew so much that our parallel grep started taking minutes instead of seconds), and it was very nice.
And then you can build your dashboards
Once you've reduced, aggregated, and indexed your events into your favourite output files and databases, you can read those databases to build very fast-running dashboards. They're fast because the data has been preprocessed in mostly-real time.
As I mentioned above, we had our pipeline reading the input files as fast as they could come in, so the receive+grind+reduce+index phase only took a few tens of milliseconds. If your pipeline isn't that fast, ask somebody why. I bet their program is written in java and/or has a lot of sleep() statements or batch cron jobs with intervals measured in minutes.
Again here, I'm not going to recommend a dashboard tool. There are millions of articles and blog posts about that. Pick one, or many.
In conclusion
Please, please, steal these ideas. Make your log and event processing as stable as our small team made our log processing. Don't fight over structured vs unstructured; if you can't agree, just log them both.
Don't put up with weird lags and limits in your infrastructure. We made 50MB/day/device work for a lot of devices, and real-time mapreduced them all on a single VM. If we can do that, then you can make it work for a few hundreds, or a few thousands, of container instances. Don't let anyone tell you you can't. Do the math: of course you can.
Epilogue
Eventually our team's log processing system evolved to become the primary monitoring and alerting infrastructure for our ISP. Rather than alerting on behaviour of individual core routers, it turned out that the end-to-end behaviour observed by devices in the field were a better way to detect virtually any problem. Alert on symptoms, not causes, as the SREs like to say. Who has the symptoms? End users.
We had our devices ping different internal servers periodically and log the round trip times; in aggregate, we had an amazing view of overloading, packet loss, bufferbloat, and poor backbone routing decisions, across the entire fleet, across every port of every switch. We could tell which was better, IPv4 or IPv6. (It's always IPv4. Almost everyone spends more time optimizing their IPv4 routes and peering. Sorry, but it's true.)
We detected some weird configuration problems with the DNS servers in one city by comparing the 90th percentile latency of DNS lookups across all the devices in every city.
We diagnosed a manufacturing defect in a particular batch of devices, just based on their CPU temperature curves and fan speeds.
We worked with our CPU vendor to find and work around a bug in their cache coherency, because we spotted a kernel panic that would happen randomly every 10,000 CPU-hours, but for every 100,000 devices, that's still 10 times per hour of potential clues.
...and it sure was good for detecting power failures.
Anyway. Log more stuff. Collect those logs. Let it flow. Trust me.
Update 2019-04-26: So, uh, I might have lied in the title when I said you can't have this logs pipeline. Based on a lot of positive feedback from people who read this blog post, I ended up starting a company that might be able to help you with your logs problems. We're building pipelines that are very similar to what's described here. If you're interested in being an early user and helping us shape the product direction, email me!
0 notes