#vertex shard unit

[livingshredder]

TIME: 9:23, 12 June 2075

LOCATION: FOREST DISTRICT, FIELDS OF GLASS, ENDLESS LINES

Exiting standby mode, X7 noticed the empty bed. It quickly noticed Cay, over in the kitchen, making herself some breakfast. Heh, it thought. Kitchens in Proxima living spaces were only ever needed with organics present. Such a strange concept to shards.

It walked over to her and wrapped its arms around her gently, recalibrating its locomotion gyroscopes for the day. She was still nude - not that she didn't care, but clothing was generally seen as unnecessary in Proxima's culture. She liked to respect that.

"Hey, what're you making?" the android asked, curiously. "Pfft, not like you'd care. Pancakes."

"Heh. Don't forget your tastes are gonna totally change once you get your new body. You'll be more interested in eating that pan than whatever you're making in it."

She laughed. "Doesn't bother me. And, hey, it's not gonna stop me having my favourites."

"So you're still sure, I take it? Haven't changed your mind yet?"

"Not at all! Trust me - I prefer the upsides. And besides. I'd look cute. Like you."

X7 blushed slightly, cyan coolant rushing into its facial thermal-convection matrix.

"Y-yeah. You would."

Cay smiled, as Vertex let her continue to cook.

"Hey, by the way, you got anything for me?", the android queried.

"Oh! Right! I grabbed you what I could for work. It's mostly last-gen hardware, but it should be more interesting than the material cubes you Shards usually get. I left the box on the table."

Sitting down at the table, X7 opened up the box - inside, various assorted computer components, as well as a couple of laptops. Not brand new, but they could've probably still fetched a fair price. "Cay-thank you! This is great!!!"

"No worries. I think I'm like, in the best place to get this sorta stuff now. Anyways, enjoy." She walked over from the kitchen; sitting down opposite Vertex and placing her plate of hot pancakes in front of her.

The lombax began to dig into the stack of pancakes, as Vertex sifted through the box of hardware, picking out one of the laptop computers. It wrapped its maw around the corner, taking a clean bite - chewing with subsequent crunches - and swallowing.

"Heh. Y'know, I feel like it's gonna be cool to be able to do that. To eat whatever I want, I mean. I envy the fact that you got to wreck that millionaire's car once. That was fucking nice."

The android blushed, mid-bite. "Y-you thought that was cool?"

"Hell yeah, you're pretty amazing. They got what they deserved."

[govindhtech]

How To Use Llama 3.1 405B FP16 LLM On Google Kubernetes

How to set up and use large open models for multi-host generation AI over GKE

Access to open models is more important than ever for developers as generative AI grows rapidly due to developments in LLMs (Large Language Models). Open models are pre-trained foundational LLMs that are accessible to the general population. Data scientists, machine learning engineers, and application developers already have easy access to open models through platforms like Hugging Face, Kaggle, and Google Cloud’s Vertex AI.

How to use Llama 3.1 405B

Google is announcing today the ability to install and run open models like Llama 3.1 405B FP16 LLM over GKE (Google Kubernetes Engine), as some of these models demand robust infrastructure and deployment capabilities. With 405 billion parameters, Llama 3.1, published by Meta, shows notable gains in general knowledge, reasoning skills, and coding ability. To store and compute 405 billion parameters at FP (floating point) 16 precision, the model needs more than 750GB of GPU RAM for inference. The difficulty of deploying and serving such big models is lessened by the GKE method discussed in this article.

Customer Experience

You may locate the Llama 3.1 LLM as a Google Cloud customer by selecting the Llama 3.1 model tile in Vertex AI Model Garden.

Once the deploy button has been clicked, you can choose the Llama 3.1 405B FP16 model and select GKE.Image credit to Google Cloud

The automatically generated Kubernetes yaml and comprehensive deployment and serving instructions for Llama 3.1 405B FP16 are available on this page.

Deployment and servicing multiple hosts

Llama 3.1 405B FP16 LLM has significant deployment and service problems and demands over 750 GB of GPU memory. The total memory needs are influenced by a number of parameters, including the memory used by model weights, longer sequence length support, and KV (Key-Value) cache storage. Eight H100 Nvidia GPUs with 80 GB of HBM (High-Bandwidth Memory) apiece make up the A3 virtual machines, which are currently the most potent GPU option available on the Google Cloud platform. The only practical way to provide LLMs such as the FP16 Llama 3.1 405B model is to install and serve them across several hosts. To deploy over GKE, Google employs LeaderWorkerSet with Ray and vLLM.

LeaderWorkerSet

A deployment API called LeaderWorkerSet (LWS) was created especially to meet the workload demands of multi-host inference. It makes it easier to shard and run the model across numerous devices on numerous nodes. Built as a Kubernetes deployment API, LWS is compatible with both GPUs and TPUs and is independent of accelerators and the cloud. As shown here, LWS uses the upstream StatefulSet API as its core building piece.

A collection of pods is controlled as a single unit under the LWS architecture. Every pod in this group is given a distinct index between 0 and n-1, with the pod with number 0 being identified as the group leader. Every pod that is part of the group is created simultaneously and has the same lifecycle. At the group level, LWS makes rollout and rolling upgrades easier. For rolling updates, scaling, and mapping to a certain topology for placement, each group is treated as a single unit.

Each group’s upgrade procedure is carried out as a single, cohesive entity, guaranteeing that every pod in the group receives an update at the same time. While topology-aware placement is optional, it is acceptable for all pods in the same group to co-locate in the same topology. With optional all-or-nothing restart support, the group is also handled as a single entity when addressing failures. When enabled, if one pod in the group fails or if one container within any of the pods is restarted, all of the pods in the group will be recreated.

In the LWS framework, a group including a single leader and a group of workers is referred to as a replica. Two templates are supported by LWS: one for the workers and one for the leader. By offering a scale endpoint for HPA, LWS makes it possible to dynamically scale the number of replicas.

Deploying multiple hosts using vLLM and LWS

vLLM is a well-known open source model server that uses pipeline and tensor parallelism to provide multi-node multi-GPU inference. Using Megatron-LM’s tensor parallel technique, vLLM facilitates distributed tensor parallelism. With Ray for multi-node inferencing, vLLM controls the distributed runtime for pipeline parallelism.

By dividing the model horizontally across several GPUs, tensor parallelism makes the tensor parallel size equal to the number of GPUs at each node. It is crucial to remember that this method requires quick network connectivity between the GPUs.

However, pipeline parallelism does not require continuous connection between GPUs and divides the model vertically per layer. This usually equates to the quantity of nodes used for multi-host serving.

In order to support the complete Llama 3.1 405B FP16 paradigm, several parallelism techniques must be combined. To meet the model’s 750 GB memory requirement, two A3 nodes with eight H100 GPUs each will have a combined memory capacity of 1280 GB. Along with supporting lengthy context lengths, this setup will supply the buffer memory required for the key-value (KV) cache. The pipeline parallel size is set to two for this LWS deployment, while the tensor parallel size is set to eight.

In brief

We discussed in this blog how LWS provides you with the necessary features for multi-host serving. This method maximizes price-to-performance ratios and can also be used with smaller models, such as the Llama 3.1 405B FP8, on more affordable devices. Check out its Github to learn more and make direct contributions to LWS, which is open-sourced and has a vibrant community.

You can visit Vertex AI Model Garden to deploy and serve open models via managed Vertex AI backends or GKE DIY (Do It Yourself) clusters, as the Google Cloud Platform assists clients in embracing a gen AI workload. Multi-host deployment and serving is one example of how it aims to provide a flawless customer experience.

Read more on Govindhtech.com

[kaguya-muneuji]

OH I HAD A FUCKING CRESCENDO IDEA (FOR THISE WHO SIMPLY DO NOT KNOW :tm: ITS OUR ENSTARS RP GROUP EHEH)

MAGICAL GIRL AU

imagine

CRESCENDO but something like Yuki Yuna is a Hero which i TOTALLY forced you to watch

the heros are the members of the “hero club” or some sort of idolistic club and they go around and sing and dance for everyone else but the main premise of the club is to find sacrifices worthy for “god”—whoever that may be. So when Crescendo’s first mission begins, they freak out.

Kiyama, although excited, has never been beyond this side of the world. Where the gods lay. His flower may be something like… a sunflower, and his weapon being a polaram or breakers or something like that. something with high mobility.

Miharu is given something like. a daisy maybe? (wasnt able to find anything on masking) and their weapon is maybe something defense-related.

Kirina’s flower is a white lily. Since Kirina expresses herself through dance her weapon might be something like a fan or some sort of leg brace based on kicking

Sato’s flower is a lilac, since Sato (i think) values family and close bonds. maybe their weapon is something based on chanting and bringing things together— like the vines itsuki has

Katsu’s flower is a Strelitzia since they symbolize a free spirit. Maybe Katsu’s weapon is a straight up electric guitar or a gun of some sorr

Shion’s flower is a white rose, my gay ass loves to focus on shion’s split between her unit and crescendo, since she can only focus on one after the events of ykyk.

Anyways with these, this idol group is tasked with the duty of protecting the world against the “Vertexes” or the wrath of the gods against humanity.

Hehe what do you think so far?

HI. I GO INSANE OVER CRESCENDO. HIIIIIIII I LOVE THEM SO MUCH DID U GUYS KNOW THAT. ANYWAY I WROTE A LOT IT GOT A BIT LONG SO. MORE RAMBLING UNDER THE CUT ^_^

@twowink @lycanthian @shards-of-brilliance @crooked-corvid hope u dont mind me tagging yall but its ur lil guys (gender neutral)

also im sorry i never got around to watching it o7 i think i accidentally closed the tab for it while doing a tab cleanup KLJHFKJSHDKFJ

also FUCK YEAH . MIHARU WITH DEFENSE. GOD. IM INSANE ABOUT THAT. IVE ALWAYS IMAGINED THEM TO BE LIKE . A PROTECTOR OF SORTS. YKNO . LIKE THEYRE CHILL BUT THEYLL ALWAYS HAVE UR BACK AND AOUR(ITDIFYGSDLIFUGSDFLKJG they have a shield and they most definitely have a helmet that they sometimes wear because im all for the "masking" and "putting on a different face" thing they have going on (if this is how you learn about this then. there u go! altho it seems i alr told you about it mostly . yeah i think i did say somehting about that) anyway. miharu sooo has paladin vibes. hgrhgh

i think kiyama (or katsu) should have gauntlets. they deserve to punch people. altho an entire electric guitar is SICKKK anyway kiyama probably would have Some armor (probably leather) on his upper body bc . hes gonna be in the middle of all these weapons ykno. he needs to be at least somewhat flexible and light on his feet right? hm. i dunno . kiyama and katsu are difficult to think about for this. katsu definitely has some lightning effects going on. That will stay.

SHION. I DIE IMMEDIATELY. OAURGHHRGHGRHGJHRGJRHG ourhg. she'd be probably one of the ones dealing dmg too. give em a sword ^_^ classic fighter. also im putting them in a skirt u cant STOP ME. oh if u havent caught on EVERYONE IS GETTING SOME ARMOR. YOU CANT STOP ME. magical girls outfits with just frills and fabric be damned. I NEED TO PUT SOME METAL ON THEM I NEED TO MAKE THEM KNIGHTS (no not en.stars knights sorryy still love them tho ^_^) GOD. IM INSANE. MAGICAL KNIGHTS WITH FLOWERS??!?!?? OUHGHHHH

heheh ok time for kiri~~ she definitely has high mobility!! aaa implementing her dancing ability into her fighting style!!!!! i think shed try to learn how to fight using a halberd at first but she finds it awkward to move around with and then she (or sm1 else helps) comes up with the idea to do what shes used to: dancing. if shes so used to moving gracefully without holding heavy things then why doesnt she just. do that ? !!!! anyway im putting her hair into a high ponytail as we speak. although im having difficulty in imagining her in armor. ill work on that.

satoo waaaa so theirs would kinda be like. immobilizing enemies? kinda?? ough interestingggggg i think their armor would be relatively light (like kiyamas) but im thinking maybeee . ok i lost the thoughts dammit its hard to think about things for sato too :((

tldr i need to draw them all. specifically i need miharu in flowy fabric and armor. with flowers. and a shield. FUCK now im thinking about hades im sorry (i am crazy ok)

[bhsdesk]

SHATTERED SKIES: THE MORNING LIGHTS Masterpost

SHATTERED SKIES: THE MORNING LIGHTS

Story and cover art by BHS

Links: AO3 | FF.net | DeviantArt | WattPad | TVTropes Page

Usagi Tsukino, Sailor Moon. Sakura Kinomoto, the Cardcaptor. Nagisa Misumi, Cure Black. Nanoha Takamachi, the White Devil. Madoka Kaname, the Law of Cycles. What happens to bring the five most legendary magical girls together from across five universes, and does even their combined strength stand a chance of preventing omniversal annihilation?

A war for existence itself has begun, and it’s all gone wrong...

A Sailor Moon / Cardcaptor Sakura / Precure All-Stars / Lyrical Nanoha / Madoka Magica mega-crossover by BHS.

Please reblog, share, read, and leave comments!

ACT I: UNRAVELING

1. Heart of Darkness (2014-10-14)

SAKURA-VERSE INVASION ARC

2. Sakura and the Strange Night (2014-10-21)

3. Sakura and the Deep Chill (2014-11-28)

4. Sakura and the End of Days (2015-02-19)

NANOHA-VERSE INVASION ARC

5. Burst (2015-04-04)

6. Break (2015-05-17)

7. Collapse (2015-06-18)

PRECURE-VERSES INVASION ARC

8. Sisters and Brothers (2015-07-29)

9. Storms and Tides (2015-08-25)

10. Fall and Rise (2015-10-03)

11. Ahead and Behind (2015-12-19)

12. Found and Lost (2016-01-13)

SAILOR-VERSE INVASION ARC

13. The Return (2016-02-02)

14. Gather Together (2016-03-26)

15. Detours (2016-05-03)

16. The Departure (2016-05-16)

MADOKA-VERSE INVASION ARC

17. Knock, Knock, Knockin' On... (2016-06-01)

18. Purgatory (2016-07-22)

19. ... In a Handbasket (2016-08-20)

ACT II: UNITING

GATHERING ARC

20. Meeting at the Crossroads (2016-10-14)

21. Recovery and Recuperation, Part I (2016-11-10)

22. Recovery and Recuperation, Part II (2016-12-11)

INTERLUDE

23. Offensive Statement (2017-01-15)

RESCUE MISSION ARC

24. Covert Ops (2017-02-20)

25. Closer Than They Appear (2017-04-07)

26. Objective Insecured (2017-05-15)

27. Reconnaissance (2017-07-21)

28. Behind Enemy Lines (2017-08-14)

29. Reinforcements (2017-10-07)

30. The Non-Canonical Lunatic Crack Chapter (2017-10-14) [Insane 3rd Anniversary Special]

31. Hostile Territory (2017-12-23)

32. Regroup and Counterattack (2018-02-12)

33. Treasonable Conduct (2018-03-01)

34. Prisoners of War (2018-04-18)

35. The Battle is Won... [EDIT]  (2018-5-25)

36. But The War Goes On (2018-6-5)

37. Fallout (2018-8-14)

ACT III: UNBREAKING

INTERLUDE

38. Master of the House (2018-10-4)

VILUY’S DATABASE:

I. Sailor Senshi, Vertex One

II. Sakura Kinomoto and Allies, Vertex Two

MIRROR SHARDS

A Shattered Skies side-story by Storyteller222

1. Shard 1 - An Awakening

2. Shard 2 - The Meeting

3. Shard 3 - The Return

4. Shard 4 - The Aftermath

5. Shard 5 - The Reaction

[livingshredder]

TIME: 23:41, 11 June 2075

LOCATION: FOREST DISTRICT, FIELDS OF GLASS, ENDLESS LINES

Somewhere in Proxima, rain fell against large, clear glass windows, as VRT-X7 - a Class-4 mobile Shard unit - wrapped its scaled arms tightly around the lombax laying next to it under the soft covers, her warmth a contrast against its colder synthetic body.

She wasn’t sleeping, not yet. It sensed she was hesitating - had something she wanted to say.

"Hey... Vertex?"

"What's up?"

"Uh... so... I've been reading up on Proxima culture. Heard you had a process for... integration."

"Yeah? What about it?" X7 sensed there was something up with her. Almost a hint of nervousness, or perhaps excitement. Still, it waited, listening.

"Well, uh... I don't know how to tell this to you, so I'm just gonna go out on a limb and say it. Vertex, I fucking hate my organic body. You know it, god knows I know it. So - I want to know. I want to know if I can go through with it."

The android faltered, taken aback - the expression of surprise visible on its face and LEDs. "Cay... you do know what that'd do to you, right? Shedding your organic body? Becoming a machine? You couldn't revert. To what you were before, I mean."

She nodded. "Yeah. I suppose it's a bit silly to think about. I've had this body for so long, it's all I know. But it sucks in so many ways. I've seen you - watched you - you don't have to deal with any of the pain."

"...You're really sure you want this?"

"Yeah. Please, Vertex. I know. I need it."

"Okay," the android replied. Holding her, it squeezed her tighter. It knew from her tone she really did mean what she said. "As long as you're okay with it. That's the main thing. How about we give it a few days - let you think it over?"

"Sounds good. Thanks. I'm sorry for dumping this all on you right now - I just needed to say it."

"It's fine, you're okay," Vertex said. And then it gently kissed the back of her neck.

“A-ah. Thanks, unit. You’re a great friend.”

Contented, the two slowly drifted into their respective thoughtspaces - the lombax into sleep, and X7 into standby mode.