#remember when there were three step sequences in the rd

[a-birdhouse-in-your-soul]

another day, another ice dance event where not a single team got a level four on any step sequence

[figure-skating-watch-party]

JGP Courchevel II

This time I – very surprisingly – decided to watch some ladies for a change so that it doesn’t seem that I live in a cave and so that I can get familiar with at least some of them.

Isabeau Levito - The Swan (Saint Seans) * Russian dance from Swan Lake I saw her at the Peggy Fleming Trophy and just knew she was special. I think a lot of people experience this “skating love” at first sight. For me, it’s Isabeau. Especially her step sequences are beautiful.

Nina Pinzarrone (BEL) - I will wait for you * Frida Nina is only 14 year old! I was amazed by her impeccable stylization into Frida. The costume, the flower crown, amazing spins, good jumps. Just an awesome program.

Kaiya Ruiter - Opportunity by Sia * Mulan I think Kaiya is an example of a skater with very balanced programs. She performs quite well for the spectators and doesn’t do many mistakes.

Ginevra Lavinia Negrello - GoT * La Terre vue du ciel She is a very expressive and mature skater. When it comes to jumps, both programs were quite unfortunate but it doens’t take away from the beautiful music and very fluid moves she does.

Niina Petrokina (EST) - Je n'attendais que vous * Love of my life  Her FP program didn’t go as planned, but I loved her great skating, mature performance and also a beautiful blue costume with red roses in the free.

And now some ice dance!

Solene Mazingue & Marko Jevgeni Gaidajenko - Clint Eastwood, Beggin * To build a home You probably already know that I’m a major hater of this season’s RD but this wasn’t bad at all. There were attempts at humorous choreography?! We take it all. And then in the FD a complete change, very nice calm program, great lift with Marko holding Solene in front of himself just by the upper part of her leg.

Barbora Zelená & Jáchym Novák - Seven Nation Army, Swing Baby * Rain in your black eyes, Fire on fire Awesome straight-line lift on one leg! That would be their trademark. But there were also 8 Ns on basic pattern :( and Jachym had some problems with twizzles (in RD as well as FD).  Good news is that Czech Republic has three junior ice dance teams!! And oh, what a great FD, it suits them much better, it’s much more fluid, the lift went even better.

Isabella Flores & Dmitry Tsarevski - Oh what a night for dancing * Nuvole bianche, Earth song Nice, oh wow what a rotational lift with Isabella hanging around Dmitry’s neck by her leg!, oh yessss, I’m happy it’s a “dancy” RD without streetdance vibes. And so fluid and light in the free, getting a silver medal and even winning the RD portion. What a debut! It’s interesting that they debuted quite late (in JGP terms) because they are both 18 yo.

Noemi Maria Tali & Stefano Frasca - Love is a lie * Always remember us this way I liked their FD, it was more lyrical with a nice spin at the beginning, it’s really soothing… And what an interesting curve lift with her “flying” under Stefano who is in a sitting position.

Oona Brown & Gage Brown - How ya like me now * Muse medley So quick, wiiiild, I liked it. But they were awarded only one yes on the blues pattern and the first half was deemed to be only basic?! That’s quite surprising, but it’s the beginning of the season. Good powerful skating in the free dance!

Eva Bernard & Tom Jochum - Black Swamp village * Kaleidoscope They are part of the Villard-de-Lans skating school and one of the many talented French junior ice dancers. This school produces some quite unconventional programs and I love it! Great blues, very defined finished moves. It’s a really enjoyable and crazy RD! And OMG the FD it’s like a mix of African woodoo and Indian tones it’s awesome. They are in synch, admittedly the music may be weird but they thrive! It was awesome. Such detailed choreography.

[takerfoxx]

Restless Thoughts

magic5ball submitted:

Sorry it took so long to get to this. Life kind of got in the way. 

Anyway, Restless. I’m gonna be honest: after mulling on this arc for a while, I’m not sure I liked it. There were individual parts that were good (and by that I mean everything with the space raptor) but overall it felt like one little thing was bogging everything down, and I think I’ve pinned down what it is:

This arc felt like a retread, character-wise. Most of the character’s traumas and insecurities faced in the chapter were things we, as an audience, already knew about or were already established in previous chapters. This wouldn’t have been so bad if I felt the characters had grown significantly from the ordeal, but the small changes that did occur (Kyoko now loving both Sayaka and Octavia) did not feel like it warranted an arc as long as this one. 

I will give credit where credit is due: the dream sequences were unique, if a bit long at times, and it is clear you put a great deal of effort into them. My only problem here is I’m not sure you achieved the tone you wanted with them. Charlotte’s private Hell, for instance, had my laughing hysterically at how tiny and pathetic she was, and I’m not sure that’s the feeling you wanted to convey.

Mephisto was interesting, if only because you made a villian whose associated color is something bright as opposed to black. It also helps that, unlike certain other authors who do this type of character, you have a degree of self awareness as to what you fanfic is and don’t treat everything super serious like you’re writing the next great american novel. 

And everything with the space raptor was gold, but you already knew that. 

There were also some technical details, but nothing major (you say the gang doesn’t set up a campfire one scene, but flames are dancing across Charlotte’s face as she describes Mephisto regardless).

Unrelated note, but I find it amusing how you describe Octavia as having chubby cheeks. For some reason, this has caused me to visualize Octavia and Sayaka as having different body types, with the former being chubbier and the latter being more muscular. 

Which, appropriately enough, was also exactly the same reaction that the original Restless got. I recall one reviewer saying that she couldn’t decide if it was a really good episode or a really bad episode, and it ended up being the only one she didn’t give a star rating to. Personally I loved the shit out of it, while other people most definitely didn’t. It just comes down to personal taste, and with something this off-kilter it stands to reason that it would be divisive.

However, I have to disagree in that this arc wasn’t plot or character relevant. Hell, as the guy writing the danged thing, I say it’s probably the most important arc yet! Firstly, this story’s been on hiatus for what, three years? Give or take? And the overwhelming majority of readers haven’t been closely following what I’ve been up to and directly interacting with me. Most read the last chapter years back and only picked the story up again when it came back. As such, a refresher arc, one that not only re-established what everyone’s personal issues are but also allowed for various callbacks to events and characters from previous arcs that might one day return was needed. I mean, show of hands, who actually remembered Brooklyn?

Besides, sometimes you just have to step back and really examine your characters under a microscope, to dig inside their heads and lay them bare.

Secondly, sure, most of it was weird dream stuff, but several important events happened during this arc that’ll continue to have an impact on the story for a long time to come, not the least of which was the introduction of the Ideal Witches. I mean, we have Mephisto now available as a possible recurring antagonist. We also have Irn, who originally played a big part in the last part of WN, making her RD debut and getting involved, and this time she’s brought a friend! What’s more, they both have resolved to involve themselves with the gang’s life, which opens things up for some very specific adventures to come.

Also, we saw the introduction of Sayaka Miki herself as an actual character! Granted, she went away soon after, but she’s officially in the story now, which is a huge game-changer in itself. And like you said, Kyoko finally accepted that she was in love with Saytavia. And Charlotte finally put her foot down on going on with the adventure any longer. And Mami’s depression, which had sort of been lurking in the background, is now coming front and center. And Oktavia, who’s only real attitude toward Sayaka was to not want to be Sayaka, now has a whole set of identity issues that just developed. 

And yeah, it did get a lot longer than I expected, but it be like that sometimes. And yeah, most of the character stuff happened in the epilogue, but in order to knock down dominos, you got to set them up first, which is what this chapter did.

Also, it was a lot of fun. 

[itbeatsbookmarks]

(Via: Hacker News)

Mike Doherty (u/Kaitaan) Senior Engineer, Data Engineering

This post is the first installment of a three-part mini-series on data architecture at Reddit, starting with Reddit’s data “origin story” and finishing with our present-day practices.

Last year, we shared a few blog posts explaining some of the interesting technical challenges we face here at Reddit. From fixing search (again), to how we route requests to different stacks, to (one of my favourites) counting the number of people who’ve viewed a piece of content, among several others, we’ve covered a very small part of what it is our technical teams do here. One area we haven’t covered much of yet is our data systems.

There’s always room for improvement in our systems, whether it’s adding more functionality, scaling to handle more throughput, or throwing everything out the window and starting from scratch. I’ve been here for all of it. And as Reddit’s very first dedicated data engineering hire, I get credit for more than my fair share of mistakes and bad decisions along the way.

This blog post will be the first of three taking you through the history of data engineering at Reddit. Over the course of the series, I’m going to talk a bit about what we did, and why we did it. More importantly, I’ll talk a bit about what problems we ran into along the way, and what we did to solve them.

During my on-site interview at Reddit, I remember asking what the current data architecture looks like. What technologies are used? What do the data pipelines and ETL systems look like? How many people are working on this, and how does it integrate into product decisions? The answers astounded me: Reddit used the free tier of Google Analytics. There were no pipelines or ETL. There was one PM who was manually running a small collection of PIG scripts on EMR, parsing HAProxy logs to get DAU/MAU numbers. This was a golden opportunity to build up data systems from scratch for a company with a massive user base, and it was an opportunity I didn’t plan to miss out on.

The early days

I still remember the first time I met our co-founder Alexis Ohanian. As I explained what I was hoping we could accomplish by having product instrumentation in place, he said to me, “Back in the early days, we just had to make all our decisions based on gut instinct.” That was a pretty good summary of how things were done back then: gut instinct. Reddit had to be careful about changes; if a change was rolled out, and users complained, we had no effective way to determine whether those complaints were a vocal minority, or were representative of all users.

The new data team at Reddit had our work cut out for us: make Reddit a data-driven company. Change product development from a world of gut-instinct change and reacting based solely on user feedback, to a world where data drives not only product ideas, but implementation, experimentation, and iteration as well. Make Reddit a data-literate company; a place where any employee can find top-line metrics, and can analyze everything relevant to their space.

Digging through access logs

Initially, we had two primary sources of data from which to try to pull insights (besides the aforementioned Google Analytics): HAProxy logs and pixel-tracking logs. These respective logs would eventually give us information about our content, such as what posts and comments receive the most engagement, as well as where, when, and on what device—with a handy breakdown of 3rd-party app traffic vs. web traffic and other critical insights into our community and products.

PIG scripts were a great start, and while they provided invaluable data at the time, we quickly realized we were going to need a more robust approach in order to get the insights we wanted. At the time, Reddit had very limited engineering resources to dedicate to this work (only one engineer), so we needed to come up with an approach that would allow us to iterate on changes quickly, add new data sources with minimal effort, and scale to Reddit’s traffic volumes right out of the gate (for context, HAProxy logs alone are more than 500 GB per day). To this end, we opted to start with a simple MapReduce job to create our first iteration of a data warehouse. This would allow us to generate basic, but more usable output than from the raw logs, and, since the raw logs were deleted after 90 days, would allow us to keep data indefinitely.

I had some experience with maintaining a Hadoop cluster in my previous job and had no desire to start down that road just yet. To do so would add a lot of operational overhead that we didn’t have the manpower to support. Furthermore, keeping a cluster up at all times when it’s only being used in bursts (daily or hourly, as logs became available to process) was also an expensive proposition. Using Amazon’s Elastic MapReduce allowed us to bring up and shut down clusters on demand, and since the data would come from and be written back out to S3, we had no need to keep a cluster up to retain data in HDFS.

Making logs queryable

In order to do any real analysis on this data, we had to be able to query it. Writing a script or a MapReduce job every time you wanted to dig in to the data to get some insights would obviously not scale well, so we needed a simple way to be able to write queries. Additionally, with the volume of raw data available, it would be nonsensical to run queries against raw data for everything.

Enter Apache Hive.

With Hive, we were able to define a warehouse and tables, and we were able to write SQL queries that would then get transformed into MapReduce jobs and run across our EMR cluster. Hive relies on a metastore to keep all information about tables (schemas, data storage locations, etc). This metastore, by default, lives on the Hadoop cluster itself, but you can define it to be in an external location. By creating our metastore on a separate MySQL instance, we kept the ability to spin up and shut down clusters on demand, which is something we do to this day (though not in EMR anymore).

Extract, Transform, Load (ETL): V1

While getting our HAProxy and pixel logs into a queryable format in Hive was the first big step, we needed some tools to run queries on a schedule, build reporting and aggregation tables, and visualize the data. Our first version of ETL used Azkaban for dependency management and Jenkins for scheduling. Jenkins is typically used for scheduling builds, but we were able to use it to talk to an Azkaban server to trigger ETL jobs. Jenkins would run a Python script on an hourly or daily basis, which would construct a SQL query or series of queries based on a set of input parameters. Jenkins would then submit the job to Azkaban, which would run the jobs and manage interdependencies between them. Once this system was up and running, Jenkins would also be configured to trigger the HAProxy and pixel processing jobs.

After so many years without a robust data system, having answers to key questions at our fingertips was great. Our new system gave us a lot of power to help determine whether we were building the right product features for our community and determine the potential impact of new ideas.

And yet, there was still one missing piece. We needed to be able to visualize the data. After all, charts and graphs are way easier to interpret at a glance than sequences of numbers!

Who wants a book with no pictures?

When we started our search, we spoke with several vendors and looked at a variety of data-visualization tools. In the end, we settled on a third-party, closed source product, largely to lighten the load of having to maintain and support something internally with our limited resources. The tool we used did not support using Hive as a backend, and given the speed of even relatively basic Hive queries, it wouldn’t make much sense to use that for reporting metrics. A dashboard that takes 10 minutes to load isn’t a particularly useful dashboard! Fortunately, a simple MySQL database on Amazon’s RDS presented a really simple solution; since the volume of data in a reporting table is small, this solution would scale for the foreseeable future, and our ETL jobs could write out there with a simple modification.

This was a great start, but the one major piece missing was making it easy for the less technical folks at the company to be able to query data. With the above system, you would have to SSH into the Hive cluster to run your queries. Wide-open access aside, that’s a painful approach for anyone. It also meant that we couldn’t scale easily to a huge number of queries. We could spin up a new EMR cluster every time, but there’s a nontrivial amount of technical overhead involved there.

Fortunately, we found a third-party vendor who provide a nice centralized system for running queries, configuring and managing clusters (Hive, in our case, but they support a number of technologies), access controls, etc. This allowed us to create a number of clusters, and anyone could log in to run their queries.

And lo, Reddit entered the Bronze Age of Data! Tune in next time for our rocky trip into the Iron Age!

Interested in joining Reddit’s growing team of engineers? Check out our Careers page for a list of open positions.

[indirispeaks]

Wichita KS Burger King

Hello, sorry I don’t have pictures but I wanted to make a post anyway.  

Yesterday, 8/3/2017 a friend and I were witness to what I am 99 percent positive was a racist incident between an cashier and two different customers at the Burger King at the shopping strip on the Kellogg (US-54) and Greenwich Rd. intersection.  (Due to the way my mental processing works I’m not good at picking up social cues or reading body language so there is the slight possibility that the employee was “just stressed” in a very stressful situation.  I was not there when it began.  On the other hand her behavior was blatant enough that I actually DID pick up on it, so it was probably exactly what I think it was.  Either way, there was NO excuse for her actions and I will be making a call to corporate as soon as I’m awake enough.)

We arrived a little after noon and there were at least 9 cars in the drive through and 11 or 12 people already inside the place waiting, including one African American woman and a Latino couple (the visibly pregnant mother was super nice) with four young children, as well as a senior Asian couple.  The rest of us were white.  Oh, and the manager of our complex and her boyfriend were there too, coincidentally.  My friend asked what was going on and the black lady informed us that they had all been told there was a staff shortage and drive through was backed up.  She, personally, had been there earlier for almost an hour, paid for her food, and did not receive it before she had to return to work.  Burger King had refused to refund her (she did not say why) so she had gotten permission from work to return and collect her food, and had been there an additional fifteen minutes. Meanwhile the warmer was half full of wrapped sandwiches

The other customers who were understandably agitated, stated that they had all been there anywhere from 15 minutes to an hour as well.  They had all placed their orders and paid, but had not received their food.......with the exception of the pregnant woman who told us that they had been there for a bit over 20 minutes and no one had yet taken her order.  She was standing in front of a register when she told us this.  There was a female young adult, white, and blonde cashier, but she was bagging food and handing it to the woman at the window.  She kept glancing over at the register but did not come over....until my friend and I stepped up to the counter in between the two registers whereupon she came over and asked what we wanted. 

.....

My friend stated that the other woman had been here way before us and to please take care of her first.  I don’t remember exactly what the bullshit excuse was, as I’ve slept since then but I vaguely recall it had something to do with their being only three of us and six of the other family.  Friend said she didn’t care as she’d just gotten here and the cashier finally took the other woman’s order and gave her cups so her kids could at least get their drinks.  Friend ordered for herself and her son.  I asked what the time estimate was and was told “Oh, not long, we’re almost through the line.”  They were not serving the customers inside because they were serving the even more agitated people in the drive through.

That dual lane drive through is such a stupid idea in my opinion, btw.  Common courtesy says that you should know to take turns because the cashiers alternate between the two order-taker-speaker-thingies but nuuuuuuuuuuuuuuu, everyone’s time is too valuable for common courtesy, so the person who ordered one or two things gets done first, no matter which lane they happen to be in, and then the car ahead of them must be tailgated as close as humanly possible to linejump them at all costs. Bonus points if you can get the car immediately behind you to do it too! It screws up the order sequence but who cares?  We got places to be!  I also think it’s a stupid idea to put a curb around the entire lane. If you’re sitting in that lane without even having gotten to the ordering bit in twenty minutes (or longer!!) and need to go, you can’t.  You’re trapped.

Anyway.

When she told us it wouldn’t be long, that was our complex manager’s cue to get up and march over to the counter.  “Excuse me, I’ve been here 45 minutes and you told me twice already that it wouldn’t be long so one, I want a manager and two, I want at LEAST a partial refund.”  The manager turned out to be the older woman at the drive thru window, who came over apologizing profusely, promising her food would be right out, and refunded everything.  “Uh.”  I could hear the other lady saying behind me.  Then the cashier came back, tossed a bag on the counter and slid it towards her without looking at her, then turned around and went back to bagging.  I decided right there I wasn’t going to give this place any money from me whatsoever.  The African American lady just sort of stood there for a second, then started to look through her bag but meanwhile my friend had blown up like an angry puffer fish, spines and all.  “MANAGER.  NOW.”  I was stuck in WTF Land.  The manager came over, with someone else’s food, and asked what could she do for her.

“I did not know it was corporate policy to skip over the first person in line to take someone’s order who just got here.”  my friend said. “Your cashier just threw her food at her.  Threw.” I said. “This isn’t what I ordered.”  the lady said.  “And I gotta get back to work.”  “I’m sorry, I’ll fix that right now.”  said the manager, as the senior Asian lady came up to the counter with the food she’d just been given.  “Sorry, I wanted these without mayo.” she said.  The manager took the tray, turned her back on the black lady still holding the wrong food, and told the guy in the back to start making whoppers right now.  After she’d just said she’d fix the other lady’s food ‘right now’

.............

“OH HELL NO.”  said Heidi. And unfortunately that was the last part I saw, because I took her son straight back to the play area.  One, he’s slightly MR and had been whining “MommyMommyMommyMommyMommyMOMMYMOMMYMOMMYMOMMYMOMMY!!” for the last ten minutes and it was aggravating people and two, I didn’t want him to hear whatever his mother was going to rain down.

P.s. She was shorted two chicken nuggets and her chicken sandwich literally had nothing on it but chicken and lettuce.

This is what I’m including in my email to corporate.  In some numbers, I am using the word allegedly because like i said, I wasn’t there and didn’t witness the whole scene.

1. Ignored pregnant Latina customer without acknowledgement. 2: Attempted to serve white customer still without acknowledging Latina customer with seniority. 3. Refunded white customer’s money after allegedly refusing to refund black customer. 4. Dishonesty about realistic wait times.* 5. Silently throwing bagged food across counter at black customer without so much as making eye contact 6. Immediately turning her back and leaving, ignoring customers 7. Didn’t even throw the correct food. 8.  Orders consistently wrong/missing/stale.* 9.  Manager did not acknowledge food-throwing. 10. Manager promised to fix black customer’s order but gave preference to another customer immediately after

*While terrible customer service, these were not part of the racism.  I’m including anyway because the entire experience was godawful in so many ways.  We were there an hour and a half.  It’s been over 24 hours and I am STILL furious.

[figure-skating-watch-party]

JGP Baltic Cup (Gdansk)

This time just a quick look at three men and six ice dance teams.

Mikhail Shaidorov (Vabank * Once upon a time in America) Spread eagle into 3A with a highkick. Oh yess! He has such an engaging short program, especially the step sequence is so much fun. Beautiful 4T-3T in the free! Maybe he was a bit slowish but that would be nit-picking. Very beautiful, calmer program with great jumps.

Andrey Kokura (Crack of Doom * Casanova) To be absolutely precise, I've seen him at Ukrainian test skates but I'd still stay that him and Vivien Papp are the JGP discoveries of the year for me. This is such a unique music and choreography. And I loved the camel spin. Oh my! Not only is his skating unique, he also has a unique costume - in the free. A shirt from a very light fabric with cut out part revealing another shirt and tattoos. I love it! Jumps were a bit misbehaving but what a performance.

Naoki Rossi (Rain in my black eyes * You raise me up) A European, small fed skater fighting for a podium? Count me interested. He'll be a very elegant skater.

Irina Khavronina/Dario Chirisano (Babylon Berlin * La cumparsita) They do crazy quick twizzles. The second part of blues pattern was so nice. I like the jazzy switch to the second half. And let me repeat: they are really so quick. In the free dance there's a lot of leg crossing and energetic moves. They are quick and they do have a personality. That's really important, especially for a Russian team when there's just so many teams to remember and to choose from.

Isabella Flores/Dmitry Tsarevsky (Oh what a night for dancing * Medley by Michael Jackson) I can say only this: It's a very pretty skating. They seem very sweet to each other. Oh my, that lift when she hangs behind his neck. This time it was a fall on the exit of the lift but it's cool nonetheless. The midline steps are awesome. I'd watch them skate it over and over. And in the free again: smooth, such pretty skating. Then came a fall out of nowhere (we can say fortunately, it wasn't on any element, so not that costly). It's a dynamic performance with interesting sliding moves.

Leah Neset/Artem Markelov (I'm feeling good, DJ turn it up * My immortal, In the end) They are a team I saw for the first time. It took me a moment to get into it but the second part of the RD is so cool: rotational lift, Leah skating under Artem, twizzles. They - along with Flores/Tsarevsky - are coached by Elena Dostatni. She, under her maiden name Khalyavina, was Maxim Shabalin's ice dance partner and they were silver and bronze JWC medalists and gold and silver JGPF medalists. (Yes, Maxim is still the most handsome and my prince) Oh what a nice, tender free dance, they beautifully in synch. And there's still time to get to know them better because they are very young.

Olivia Oliver/Joshua Andari (Ain't no sunshine * Stairway to heaven) Good performance in the RD, yes! Poland will need some talents so that Kaliszek/Spodyriev aren't alone there. Expressive, slow movement in the FD, they let you experience the dance at the beginning and then  the music picks up tempo in the second half.

Chaima Ben Khelifa/Everest Zhou (Golden Eye, Die Another Day * Keeping you alive) The posture! The regal way they carry themselves. I love it. Maybe they are a bit slower. But also the dance is a bit calmer and not so party/dance-y which is also good to have some more variety. There's a beautiful curve lift at the beginning of the free. And the rotational lift was also great, good twizzles, yes they are good.

Giorgia Galimberti/Matteo Libasse Mandelli (No diggity, Crazy in love, Yeah * Moonlight sonata) They had this modern touch in their last season's programs so I thought this RD would fit them and I think I wasn't wrong. Oh yess, that's their FD from the last season! They showed in only in domestic comps. It's surprising, it's fresh. Yes, of course they are not so quick and smooth, and this time they had some problems with the twizzles but it's still nice to watch the choreography.