#obviously sample size means that you usually just experience a very small number of things from the other countries
Explore tagged Tumblr posts
Text
Having food stuff from other countries is so much fun especially when you notice differences to things you're used to but never questioned before
#obviously sample size means that you usually just experience a very small number of things from the other countries#but for example I have some tea and sweets I got from an american person as small favors accompanying what I bought from them#so they thought it was a good thing to share right?#I mean obviously sometimes I can buy foreign stuff at the store#but I can't really know if that is something people there actually eat#or if it's just something marketed that way#but these sweets are actually from someone that bought them over there!#I think that's really fun#some things I want to try because I'm curious are cream soda and irn bru#they both sound really interesting and we don't have an equivalent here#sadly they're also not really available lol
1 note
·
View note
Text
OP is spot on really, I have no real quibbles. Also without doing any further digging, OPs statement about having to explain hits at work makes me suspect they've got some background. But I'm a Stats Person™️ and a big big fanfic reader (from ye olden days of live journal and pairing specific web forums even) so I have some further thoughts I want to share lol
1) I do actually have a quibble, it's about the law of large numbers. And the idea that it particularly matters how things like hits are calculated, or people's intentions with their kudos.
Basically, "law of large numbers" says that the larger the sample size the more it trends towards the average. Ie the more times you flip a coin the closer you will get towards 50/50 heads vs tails distribution. This is related to the hits/kudos thing because no matter how ao3 measures hits, either the anomalous bff checking the fic 10 times in a day will disappear into the general background noise, or other fics will experience roughly similar things often enough to balance those anomalies out. Likewise with kudos - generally speaking people leave kudos as a indicator of some positive feeling toward a fic, so anomalous trivial or "pity" kudos will trend towards being insignificant.
That being said -- ultimately I think OP was primarily talking to fanfic /authors/ not readers.... The thing about the law of large numbers is that it only works.... In large numbers. So as a reader looking for a very rough heuristic to start evaluating fic from, sight unseen, using these stats is great! And the more fic in the pool I'm looking through, the more generally accurate it is going to feel. If a really really good fic has an abysmal ratio.... That doesn't really matter to me as a reader? I mean, it does because I want to read that fic, but since I don't actually know for sure it exists I'm not actually losing anything by just looking at the top 200 fics according to my heuristic -- I don't have enough time in my day to check out every fic so I'm going to miss some gems no matter what I do, and if my heuristic /tends/ to find me fics I like it's worthwhile to use, even if it's not perfect.
But for looking at a specific fic, ie an author evaluating a fic they wrote, the numbers are meaningless, just like OP was talking about. Those various anomalous scenarios hit /hard/ on any given single fic.
OK! Also I want to talk a little about my own methods for finding fic, now that's all out of the way. Because that law of large numbers stuff is relevant to my methods lol.
So - basically I have 3 main ways of finding fic. 1 is non-actionable -- that is, it's finding fic recs on social media or from friends, etc. purely passive on my part, nothing more to say about it.
2) like OP mentioned - look at the other fics of authors you like, look at the bookmarks/kudos of authors you like (people generally, broadly speaking, tend to like the kind of stuff they read? Obviously there are exceptions, but in my experience this tends to be the case. If you want to find whump for a certain pairing, check the bookmarks of a whump writer for that pairing, etc) also check the bookmarks of people who commented (or left kudos, but that's usually a less significant indicator).
3) heuristics baby -- I'm lazy so I don't generally bother to actually sort by ratio, but if I'm jonesing for some specific pairing/tag combo I have a process:
- click on the pairing tag (this will be the basis of my search)
- in the filter sidebar, include/exclude whatever additional warnings, tags, rating level, etc as suits my particular desire de jour
- here comes a transition point -- either this is a target rich environment (>2000 fics, ie 100 pages), a small bool (<200 fics, ie 10 pages), or somewhere in between.
- if it is a small pool, I generally just sort by update date and read summaries to find fic I might like
- if it is a target rich environment I generally consider adding more include/exclude restrictions, probably a word count limit ( I like longer fic which cuts out a lot ), probably remove crossovers (mostly to get rid of incredibly annoying compilation "fics"), possibly throw in a >X kudos where X is picked to reduce the number of fics to ~500, and then sort by hits asc (ie least hits first). I'll also sometimes do a moving date window - ie sort by kudos, in the last 6 months, then when I stop finding fic I like move it back another 6 months, etc; especially for fandoms with a long active period, like Supernatural, Batman, a lot of classic anime fandoms, etc
- if it's in between... Depends on how much patience I have that day for "less approachable" fic (ie fic where the prose is further from the sort of "house style" people talk about these days, or fic that strays further from fandom tropes, etc) if I am feeling up to expending more effort in my reading I tend to just go by update date. If I need something less chewy then I'll just sort by kudos desc (most to least)... It's not the most accurate heuristic, but it's very broadly speaking applicable
Some notes:
The initial "weeding" is Very Important -- ie, getting your initial pool of fic. If this pool is too broad, then fandom trends or differences in community behavior can throw stuff way out of whack. IE if you particularly want to read gen fic, just searching the "obi-wan & anikin" tag is NOT likely to get you effective results... So you've got to exclude what you don't want to see -- and given the way tagging tends to be an "err on the side of including it" thing, then you are likely going to exclude some fic you would have really liked actually! And if you leave some ship tags in, it is my experience that ship fic often gets more engagement then gen fic.... So if you use the quick and dirty heuristics to give you a starting point, probably you'll have to do more "wading" at the top of your list then is really the intention of using kudos/comments as a sorting function.
If you have multiple fandoms included in your search results -- maybe you are particularly interested in a certain trope, whatever -- the more likely you'll see weird stratification of the results.... Just because some fandoms are waaaay bigger then the others. Or even just chattier? I don't have any hard numbers, but I'm pretty sure that some fandoms have waaaay different numbers vis a vis common hits/kudos/comments ratios just because of like. Fandom culture stuff. There is a certain sweet spot where the fandom is big enough to be popping, but small enough that people tend to be chatty with each other that I think gets dropped with the super large fandoms which tend to have a lot of pure lurkers -- who might kudos in droves, but never ever comment. So if you are searching a particular trope tag by comment/hit ratio, you will probably end up with weird "bands" of fandom -- which don't necessarily compare to each other quite right. YMMV on if that matters to you at all tho, and it's easily compensated for by just restricting your search to one fandom at a time (same thing happens for pairings within fandoms, but generally to a lesser extent)
-- tl;dr: OP is totally right, that being said here is a very long description of some ways I find fanfics lol
I see a lot of posts going by about comments and kudos and hits and...well... I've been thinking about the three quite a lot lately--as both a fic author and someone who spends a lot of my professional life looking at web metrics and determining which are actually important/accurate measures of user engagement.
Mileage varies, of course. And this is all just MY opinion, so do feel free to ignore it wholesale.
What I think when I see someone say that sorting by a hits to to kudos ratio is a good way to find "good" fic:
Hits are a measure of quantity (how many times your story or art has been viewed), but without knowing how AO3 defines a hit, it's actually kind of a meaningless number. We know that our own views of our work do not count toward hits, but...if my BFF looks at my story 7 times in one day because she keeps trying to read it but getting interrupted...is that one hit, or seven? And if it's seven, then the numbers are artificially inflated because it's really just Bestie trying to get her Codex fix. And...if Bestie looks at it three times today and four tomorrow...is that 7 hits total, or two?
Some transparency on the part of AO3 could clear this up handily, but until we get that...shrug. All it is is a number that may or may not be an accurate reflection of how many actual people looked at the page your fic is on. Did they READ it? Or did they nope out? No way to know.
Kudos are intended to be slightly more qualitative, but there is no way of knowing why the reader gave them. (Similar to likes here on tumblr.) It might be that they loved the piece. It might be a simple acknowledgement that the reader was there. It might even be a pity kudo. We have no way of knowing. It's, again, just a number.
Obviously, everyone is free to interpret both hits and kudos as positive reaction/interaction. I might do that myself if I didn't spend my workdays explaining to people that 50,000 "hits" to the website could be 50K people who came to learn about us or...simply the result of the computer labs on campus having the university homepage set to default.
Bigger numbers are just that....bigger numbers.
Comments are the only objective way to judge how someone is reacting to your fic or art.
So, what then? Sort by number of comments?
You can do that, sure. (I think. I confess I have never once gotten the AO3 search to work as well as people rave about.) But do keep in mind that many authors answer their comments. So, something with, say, 20 comments may be 20 people telling the author they loved it. Or it might be ten people and ten author-replies. OR, it might be three people having a conversation in the comments. You have to look and see.
Bigger numbers are just bigger numbers.
Okay, fine Elis. What am I supposed to do then?
Look, I'm not your mother or your therapist and you are free to assign whatever meanings you like to these things. I, personally, find "good" fic through a combination of things including: recs, the fandom grapevine, dumb luck, events, and just...reading some of it and not feeling guilty if I nope out for some reason.
This all sounds a little depressing when laid out like this, huh? Especially when you take into account the downward trends in interacting and the rise of folks treating fic and art as content to be consumed.
Here's what I have learned from writing fic for 30 years (well, 28 and counting):
As an author (and an artist, I would presume), you have absolutely no way of predicting which of your work will land and take hold and which will not. It's alchemy and luck and the weird (and not actual) algorithm of fandom. Sometimes, the piece you whipped out in 30 minutes and posted on the fly will land in the right person's inbox and they will share it and their friends will share it and it will get big. Sometimes, the piece you slaved over for weeks and weeks will do that...sometimes it won't. Sometimes your genius manifests and resonates, sometimes it does not.
My personal favorite fic of my own--the one I think is probably the best thing I have done in SW fandom-- has like 8 kudos and 4 comments (2 of which are my responses). Is it disappointing? Yes. Is it an indication that the fic is objectively "bad"? No.
The mercenary in me suggests that if you want to get lots of comments and kudos, you should pick the pairing that is THE pairing in the fandom and write for that--because that's where the eyeballs are, because that's where the connections are. But that is not why I write, so it's just that--a very mercenary way of looking at things. Not that there is anything WRONG with doing it that way. Supply and demand run the world. If the people want Codywan and you want the people....give them Codywan. No shame in that.
And there is no shame in wanting or seeking validation for your work, either.
But it breaks my heart to see authors (and artists) give up on themselves when they do not receive piles of kudos and comments. It's not you. It's...the luck of the draw. It's...fandom. It's...an artificial and murky set of measurements that have almost no basis in anything meaningful.
Keep writing. Keep drawing. Keep sharing. You are what you make, not how people respond to it.
81 notes
·
View notes
Note
CULT WIP Hi there~! So I'm working on a cult WIP and I know a common way to control people in a cult is sleep deprivation. I've looked through your sleep related tags and the cult one, but right now I'm wondering about the long con of sleep deprivation. My idea is that to keep people controlled they get less sleep than they should (your posts have made me up it from three hours per night to five per night) and the higher ups in the cult are allowed the full amount of sleep they need; (1/2)
CULT WIP (2/3) that way they operate better than others. I've jotted down a bunch of effects you've brought up but here's the thing. What if that sleep deprivation lasts forever? Like it's just a normal part of life once you hit adulthood? (I don't think kids could withstand it so I figured just not to do it). I know the Heaven's Gate cult used a lot of sleep deprivation and they lasted for ~three years so there must be some way to/balance to keep people functional, I just can't think of how.CULT WIP (3/3) Not actually part of the question but I just wanted to say THANK YOU for this blog!! It's fascinating, especially for me since I'm in grad school to be a therapist so that aspect is really interesting, and you put an incredible amount of work into all this. ~Jessica
Thankyou, it’s always nice to hear I’ve been helpful. :)
I’mgoing to go into this one with the caveat that so far as I know, noone has done this experiment. There isn’t a studied group of peoplewho have been restricted (or restricted themselves) to five hours ofsleep a night every day for their entire adult life. There are peoplewho’ve done this for a number of years and people who do this 5-6nights a week and then sleep more on the weekend for decades at atime. Now all three situations are bad for someone, but a thoroughstatistical analysis on a decent sample size might show differencesbetween them.
Sopart of this is what we know happens to the first two groups and partof it is extrapolating based on that.
Thefirst thing we knowhappens (based on the two studied groups) is a shorter, unhealthierlife.
Ihesitate to put a figure on how much shorter because it seems to varyquite a bit between individuals and I don’t know of any statisticalstudies that have put a number on it. But I think you can safely takeat least ten years off every character’s life expectancy based onthis alone.
Therates of a lotof different diseases and conditions increase. Cancer rates rise, formultiple forms of cancer. Rates of strokes and heart attacks rise.Dementia rates rise.
Nowall of those conditions are usually diseases of age. You canhave a heart attack or a cancer diagnosis as a young person, but thechances of it happening are much more likely after you hit about40-50.Lack of sleep doesn’t seem to effect the age these conditionsmanifest. It doeseffect the chances of them happening in vulnerable ages though.
Essentiallyif you take a group of 40-50 year old non-cultists from your valleysetting (I hope you don’t mind me looking at your blog? Lovelypictures by the way) less of them will have or have had cancer,strokes, heart attacks and early signs of dementia. As the populationages further the gap will become starker. Less of the cultists willsurvive to their 70-80s and those that do will be less healthy thenthe non-cultists.
Diabetesrates also increase with lack of sleep. This doesn’t appear to beage related. It is however unclear whether it’s because of theeffect lack of sleep has on the immune system or because of theeffect it has on our appetites. People who sleep less eat more andtheir bodies drive them towards more high fat and high sugar foods.
Idon’t understand the link between weight and diabetes very well, soI’m not going to talk about it in any depth. The general point I’mdriving at is that if your cult tightly controls diet that mightcounteract the rise in predisposition to diabetes. But the data isn’tentirely clear on that point.
There’salso a general rise in illness and infections. That contributes todecreased life expectancy but also means more sick days. Less timewhen any one individual can productively work.
Partof what this is gearing towards is this: I’m not sure it would bepossible to consistently keep someone on five hours sleep a nightonly for their entire life without a huge death rate.
It’sthe illnesses. I think if cult members were denied sleep while sick(especially if they’re also forced to work or their diet iscontrolled) then- well I think there’d be a lot more people dyingfrom common, preventable illnesses. Not instantly. Not within thefirst decade. But in the longer term or thirty or so years.
Onthe other hand if the cultists who are ill aregetting enough sleep then you don’t strictly have five hours sleepa night for the rest of their lives. What you’ve got instead issomething more like ‘five hours sleep a night until you reachphysical collapse, then you can rest’.
That’sextremely unhealthy, painful and harmful. But it’s less likely tokill so many people so quickly.
Partof the issue is how ‘functional’ the characters need to be. Atfive hours, it would be dangerous to drive or operate other heavymachinery. Accidents would be more likely. Mistakes would be morelikely.
Butthat doesn’t mean these characters couldn’t do most of the day today tasks required to keep a small community going.
It’snot that the cooking couldn’t get done, it’s that the chances ofdropping a pan full of boiling water on someone’s foot is a lothigher. Less that complex tasks can’t get done and more that they’dtake longer, be completed less well, less effectively and there’dbe a higher chance of accidents on the way.
Incidentallyif a big part of this story is the standard tactic of elite membersof the group making other members feel less confident in themselves-usethis effect of sleep deprivation to help accomplish that.Because people who are sleep deprived thinkthey are physically and mentally capable of more than they are.
Youcan sit them down and say ‘Listen S, the low amount of work you’vegotten through this month is unacceptable. We agreed that you couldfinish this project in a week and it’s taken two. You’re just nottrying hard enough’
Andtheywill agree.Because they don’t know how impaired they are. It’sone of the stranger effects that consistently shows up in testing andI feel like it’s very relevant here.
Theincrease in workplace accidents is also affected by the decrease inimmune function. Accidents are more likely andrecovery from them takes longer.
Theother thing that stands out to me is the effect this would have onthe living environment generally.
Sleepdeprived people are emotionally volatile. They also tend towardsbeing distrustful of others and paranoia. Again this isn’tsomething they necessarily recognise.
Whichmakes for a pretty horrendous environment when you think about alarge group of people living in fairly close quarters andunable to really avoid each other.
Thinkabout how this meshes with the rise in accidents, forgetfulness andgeneral tiredness that go along with sleep deprivation and you’llsee what I mean. Someone drops the hot pan and it just misses someoneelse’s foot- was it deliberate? Someone forgets where they putsomething- obviously it was stolen. There was a stray shoe left outin the hall and a character almost tripped over it- clearly whoeverleft it there knew thatcharacter could/would trip.
Andso forth.
Fromthe point of view of your cult leaders this sort of misery andemotional upheaval is a positive. It makes it harder for people toorganise or relate in an authentic way to each other in the longterm. It couldmake it especially hard for parents and children to keep up apositive relationship, comparedto the relationship the children could have with the cult leaders.
Becausethe parents will always be too tired, too grumpy, too unpredictable,to relate well to a young child. Whereas the well rested cult leaderscould appear calmer, kinder andseem to have more time.
There’svariation within all of this obviously. Despite damagingcircumstances some people do live to a ripe old age and don’tdevelop cancer more than ‘normal’ people would (chemistryprofessors over 80 are an interesting breed). Some people may stillbe able to show some patience and kindness despite the effects sleepdeprivation has on emotional regulation.
Moneyand treatment can also extend the life of someone who is routinelysleep deprived and suffers from multiple health problems as a result.
Ifyou’ve read my previous posts then I think you’ll have an idea ofindividual symptoms and how they get progressively worse. A lot ofthis ask was me- not just trying to map out what the indefinite timeframe you have would look like but the effect it would have on agroup and the relationships within that group.
Ihope that helps. :)
Availableon Wordpress.
Disclaimer
#tw torture#tw cults#cults#sleep deprivation#effects of sleep deprivation#effects of torture on organisations#scripturient-manipulator
23 notes
·
View notes
Photo
![Tumblr media](https://64.media.tumblr.com/939d92cab348781ad96dfc1c6dc37bf1/tumblr_pxsnvpM2Vk1rcf34no1_540.jpg)
The Truth About Electric Eels Has Long Been Overlooked
I’m shocked, shocked, I tell you.
ED YONG
SEP 10, 2019
Indigenous people in Venezuela called it arimna, or “something that deprives you of motion.” Early European naturalists referred to it as the “numb-eel.” And for 250 years, since it was first given a Latin name, Western scientists have known it as Electrophorus electricus, the electric eel, the sole member of its genus, the first and only of its name.
Throughout the animal’s storied history—as Alessandro Volta designed the first synthetic battery based on its body, as Alexander von Humboldt collected it by driving horses into eel-infested pools, as a young Charles Darwin dissected the creature aboard the HMS Beagle, as the physicist Michael Faraday placed his bare hands on it in his quest to understand electricity, and as modern researchers carried out an array of studies to show just how amazing (and sinister) its abilities are—Electrophorus electricus has always been regarded as a single species. The electric eel.
Carlos David de Santana, a Brazilian researcher at the Smithsonian National Museum of Natural History, thinks differently. By comparing 107 specimens pulled from museum drawers and the Amazon basin, he and his team of mostly Brazilian scientists have found that the infamous electric eel is actually three distinct species.
There are dozens of different ways of defining a species, and none are universally accepted. That said, de Santana says that his team “used many lines of evidence to prove that there’s more than one electric eel species.” This trinity differs not only in physique, but also in genetics, habitat, and electric power. Tellingly, the eels’ DNA suggests that they last shared a common ancestor 7 million years ago, which means that they started to diverge well before brown bears and polar bears, lions and tigers, and even humans and chimpanzees.
One of the trio retains the original name Electrophorus electricus, and de Santana now calls it Linnaeus’s electric eel, after the legendary Swedish taxonomist who classified it. The two others are now Volta’s electric eel (Electrophorus voltai), after the Italian physicist who built a battery based on the animal, and Vari’s electric eel (Electrophorus varii), after Richard Peter Vari, a famous ichthyologist who was part of de Santana’s team until his death in 2016. (Most of the eels used in previous research are likely to be Vari’s eels, since they’re the only species from Peru, the only country from which these animals can be legally exported.)
“These findings do not surprise me,” says Graciela Unguez from New Mexico State University. As researchers sample electric fishes from more parts of South America, she adds, they’re almost bound to find that currently known species harbor more diversity than people suspected.
The same goes for unusual animals whose outward distinctiveness can mask subtler differences that become clear only through genetic analyses. Such studies have shown that there are likely four distinct species of giraffe, three species of mola mola, and two species of African elephants. “We sort of lump the weirdos together,” says Prosanta Chakrabarty from Louisiana State University. “Oh, obviously, this thing is that thing, and no one looks more carefully. We all thought that an electric eel is an electric eel.”
Contrary to appearances, they’re not even eels. They’re knife fishes—a group of mostly small, gill-breathing species that have flattened bodies and that produce weak electric fields for navigation and communication. The misnamed eels buck all these trends—weirdos, even within their own family. They breathe by rising to the surface and gulping air, which makes them one of the only fish that you can drown. Their thick, cylindrical, meaty bodies can reach seven feet in length. And the electric organs that make up 80 percent of that length can produce shocks that are strong enough to incapacitate a human or a horse.
Collecting these animals from the wild, as de Santana did, is not easy. “I do it by myself, or with the help of really experienced fishermen,” he says. “I don’t allow students to do it. It’s never safe.” Even if he wears rubber gloves, the sweat that builds up inside them eventually links up with the water outside them, creating a continuous conductive layer. Bottom line: You can’t collect electric eels without suffering shocks, which de Santana compares to getting hit with a Taser. It’s even worse in the dry season, when more than 10 individuals can occupy a single stream. “When one starts to discharge, the others do too,” says de Santana. “You just get used to it. You do what you have to do.”
Once the samples were in, the team focused on 10 important genes. Immediately they saw that the eels clustered into three distinct groups, with very little genetic variation within each one, but substantial genetic differences between them. In one key gene, for example, the three species differ by 6 to 10 percent of their DNA, but individuals within each species differ by 0.3 percent at most.
Looking closely, the team realized that there are physical differences among these three species—not in obvious features such as size or color, but in subtler ones, like the flatness of their heads, or the number of pressure-sensitive pores on their flanks. With experience, de Santana can now tell the three species apart by eye.
In the wild, it’s even easier: The three eels live in different habitats, which might explain why they’re distinct. About 7 million years ago, some ancestral electric eel split into two populations. Vari’s eel lives in lowland floodplains, whose waters are usually murky, muddy, and oxygen-deprived. The two others live in highland rivers, where water is fast-flowing, well oxygenated, and clear. Though they share the same environment, their ranges don’t overlap: Linnaeus’s eel is restricted to northern Amazonia, while Volta’s eel lives in the south.
What separated them? Most likely, the Amazon River itself. Around 9 million years ago, after eons of flowing westward, the mighty river started reversing its course. Its modern eastward flow became entrenched around 2.5 million years ago—exactly when Volta’s and Linnaeus’s eel split into distinct species.
These different habitats have likely influenced the animals’ use of electricity. Clear water contains fewer dissolved minerals than muddy water, and is worse at conducting current. So to stun their prey, Volta’s and Linnaeus’s eels either need to get closer than Vari’s eel does or release stronger shocks. Volta’s eel certainly does the latter: De Santana’s team found that it can discharge up to 860 volts. That’s far higher than the 650 volts commonly cited for electric eels, and beyond the abilities of any other electric fish.
The three species might also behave differently. It’s commonly said that electric eels are solitary hunters that use electricity to locate prey in murky water, but de Santana’s team has evidence that the two clear-water species live in groups and hunt collectively.
These discoveries, made largely in Brazil and by Brazilian scientists, come at a difficult time for the nation’s researchers. The National Museum in Rio de Janeiro—the largest natural-history museum in Latin America—was gutted by a fire last year, destroying millions of priceless specimens in a preventable tragedy caused by inadequate funding.
The electric eels’ wild habitat is also on fire. About 40,000 blazes have swept through the Brazilian Amazon this year—an 80 percent rise from last year. Most of these were deliberately ignited to make way for agriculture by burning out forested lands, and the indigenous communities living there. That arson has been tacitly encouraged by Brazil’s far-right president, Jair Bolsonaro, who promised to undermine protections for the Amazon, open it up for economic development, and wrest control of land from indigenous groups. “It’s a really bad situation,” says de Santana, who is Brazilian himself. “I go to the Amazon twice a year. From what I’ve seen, I’d say that in 50 years’ time, we’ll only have fragments of what we have today.”
The electric eels should serve as reminders of what could be lost as the Amazon shrivels and smolders. “They’re eye-catching animals that have been known for 250 years, and that live in one of the Earth’s biodiversity hot spots,” says de Santana. “If you can find new species like that, what else could you find there?”
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast: We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°). How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords. This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4: Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate. The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency: Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story: If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone. Now, let’s pick three different data points (all of these are from the top 20): From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes. There’s an even weirder story buried in the May 2020 data. Consider this: LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update): Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%. Now let’s look at Google Play, which appeared to be a clear winner after two days: You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update. How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means: While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history. Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions. Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers: Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change: Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains. Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after: It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions. Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time. Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains). Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
https://www.businesscreatorplus.com/googles-may-2020-core-update-winners-winnerers-winlosers-and-why-its-all-probably-crap/
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
via Blogger https://ift.tt/2WsXYgc #blogger #bloggingtips #bloggerlife #bloggersgetsocial #ontheblog #writersofinstagram #writingprompt #instapoetry #writerscommunity #writersofig #writersblock #writerlife #writtenword #instawriters #spilledink #wordgasm #creativewriting #poetsofinstagram #blackoutpoetry #poetsofig
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
https://ift.tt/2AjddzJ
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast: We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°). How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords. This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4: Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate. The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency: Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story: If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone. Now, let’s pick three different data points (all of these are from the top 20): From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes. There’s an even weirder story buried in the May 2020 data. Consider this: LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update): Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%. Now let’s look at Google Play, which appeared to be a clear winner after two days: You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update. How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means: While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history. Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions. Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers: Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change: Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains. Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after: It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions. Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time. Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains). Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
https://www.businesscreatorplus.com/googles-may-2020-core-update-winners-winnerers-winlosers-and-why-its-all-probably-crap/
0 notes
Text
Google's May 2020 Core Update: Winners, Winnerers, Winlosers, and Why It's All Probably Crap
Posted by Dr-Pete
On May 4, Google announced that they were rolling out a new Core Update. By May 7, it appeared that the dust had mostly settled. Here’s an 11-day view from MozCast:
We measured relatively high volatility from May 4-6, with a peak of 112.6° on May 5. Note that the 30-day average temperature prior to May 4 was historically very high (89.3°).
How does this compare to previous Core Updates? With the caveat that recent temperatures have been well above historical averages, the May 2020 Core Update was our second-hottest Core Update so far, coming in just below the August 2018 “Medic” update.
Who “won” the May Core Update?
It’s common to report winners and losers after a major update (and I’ve done it myself), but for a while now I’ve been concerned that these analyses only capture a small window of time. Whenever we compare two fixed points in time, we’re ignoring the natural volatility of search rankings and the inherent differences between keywords.
This time around, I’d like to take a hard look at the pitfalls. I’m going to focus on winners. The table below shows the 1-day winners (May 5) by total rankings in the 10,000-keyword MozCast tracking set. I’ve only included subdomains with at least 25 rankings on May 4:
Putting aside the usual statistical suspects (small sample sizes for some keywords, the unique pros and cons of our data set, etc.), what’s the problem with this analysis? Sure, there are different ways to report the “% Gain” (such as absolute change vs. relative percentage), but I’ve reported the absolute numbers honestly and the relative change is accurate.
The problem is that, in rushing to run the numbers after one day, we’ve ignored the reality that most core updates are multi-day (a trend that seemed to continue for the May Core Update, as evidenced by our initial graph). We’ve also failed to account for domains whose rankings might be historically volatile (but more on that in a bit). What if we compare the 1-day and 2-day data?
Which story do we tell?
The table below adds in the 2-day relative percentage gained. I’ve kept the same 25 subdomains and will continue to sort them by the 1-day percentage gained, for consistency:
Even just comparing the first two days of the roll-out, we can see that the story is shifting considerably. The problem is: Which story do we tell? Often, we’re not even looking at lists, but anecdotes based on our own clients or cherry-picking data. Consider this story:
If this was our only view of the data, we would probably conclude that the update intensified over the two days, with day two rewarding sites even more. We could even start to craft a story about how demand for apps was growing, or certain news sites were being rewarded. These stories might have a grain of truth, but the fact is that we have no idea from this data alone.
Now, let’s pick three different data points (all of these are from the top 20):
From this limited view, we could conclude that Google decided that the Core Update went wrong and reversed it on day two. We could even conclude that certain news sites were being penalized for some reason. This tells a wildly different story than the first set of anecdotes.
There’s an even weirder story buried in the May 2020 data. Consider this:
LinkedIn showed a minor bump (one we’d generally ignore) on day one and then lost 100% of its rankings on day two. Wow, that May Core Update really packs a punch! It turns out that LinkedIn may have accidentally de-indexed their site — they recovered the next day, and it appears this massive change had nothing to do with the Core Update. The simple truth is that these numbers tell us very little about why a site gained or lost rankings.
How do we define “normal”?
Let’s take a deeper look at the MarketWatch data. Marketwatch gained 19% in the 1-day stats, but lost 2% in the 2-day numbers. The problem here is that we don’t know from these numbers what MarketWatch’s normal SERP flux looks like. Here’s a graph of seven days before and after May 4 (the start of the Core Update):
Looking at even a small bit of historical data, we can see that MarketWatch, like most news sites, experiences significant volatility. The “gains” on May 5 are only because of losses on May 4. It turns out that the 7-day mean after May 4 (45.7) is only a slight increase over the 7-day mean before May 4 (44.3), with MarketWatch measuring a modest relative gain of +3.2%.
Now let’s look at Google Play, which appeared to be a clear winner after two days:
You don’t even need to do the math to spot the difference here. Comparing the 7-day mean before May 4 (232.9) to the 7-day mean after (448.7), Google Play experienced a dramatic +93% relative change after the May Core Update.
How does this 7-day before/after comparison work with the LinkedIn incident? Here’s a graph of the before/after with dotted lines added for the two means:
While this approach certainly helps offset the single-day anomaly, we’re still showing a before/after change of -16%, which isn’t really in line with reality. You can see that six of the seven days after the May Core Update were above the 7-day average. Note that LinkedIn also has relatively low volatility over the short-range history.
Why am I rotten-cherry-picking an extreme example where my new metric falls short? I want it to be perfectly clear that no one metric can ever tell the whole story. Even if we accounted for the variance and did statistical testing, we’re still missing a lot of information. A clear before/after difference doesn’t tell us what actually happened, only that there was a change correlated with the timing of the Core Update. That’s useful information, but it still begs further investigation before we jump to sweeping conclusions.
Overall, though, the approach is certainly better than single-day slices. Using the 7-day before-vs-after mean comparison accounts for both historical data and a full seven days after the update. What if we expanded this comparison of 7-day periods to the larger data set? Here’s our original “winners” list with the new numbers:
Obviously, this is a lot to digest in one table, but we can start to see where the before-and-after metric (the relative difference between 7-day means) shows a different picture, in some cases, than either the 1-day or 2-day view. Let’s go ahead and re-build the top 20 based on the before-and-after percentage change:
Some of the big players are the same, but we’ve also got some newcomers — including sites that looked like they lost visibility on day one, but have stacked up 2-day and 7-day gains.
Let’s take a quick look at Parents.com, our original big winner (winnerer? winnerest?). Day one showed a massive +100% gain (doubling visibility), but day-two numbers were more modest, and before-and-after gains came in at just under half the day-one gain. Here are the seven days before and after:
It’s easy to see here that the day-one jump was a short-term anomaly, based in part on a dip on May 4. Comparing the 7-day averages seems to get much closer to the truth. This is a warning not just to algo trackers like myself, but to SEOs who might see that +100% and rush to tell their boss or client. Don’t let good news turn into a promise that you can’t keep.
Why do we keep doing this?
If it seems like I’m calling out the industry, note that I’m squarely in my own crosshairs here. There’s tremendous pressure to publish analyses early, not just because it equates to traffic and links (frankly, it does), but because site owners and SEOs genuinely want answers. As I wrote recently, I think there’s tremendous danger in overinterpreting short-term losses and fixing the wrong things. However, I think there’s also real danger in overstating short-term wins and having the expectation that those gains are permanent. That can lead to equally risky decisions.
Is it all crap? No, I don’t think so, but I think it’s very easy to step off the sidewalk and into the muck after a storm, and at the very least we need to wait for the ground to dry. That’s not easy in a world of Twitter and 24-hour news cycles, but it’s essential to get a multi-day view, especially since so many large algorithm updates roll out over extended periods of time.
Which numbers should we believe? In a sense, all of them, or at least all of the ones we can adequately verify. No single metric is ever going to paint the entire picture, and before you rush off to celebrate being on a winners list, it’s important to take that next step and really understand the historical trends and the context of any victory.
Who wants some free data?
Given the scope of the analysis, I didn’t cover the May 2020 Core Update losers in this post or go past the Top 20, but you can download the raw data here. If you’d like to edit it, please make a copy first. Winners and losers are on separate tabs, and this covers all domains with at least 25 rankings in our MozCast 10K data set on May 4 (just over 400 domains).
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
0 notes