All We Need Is Ratio Ga-Ga

ListenersDesc.png
Figure 01: My artist-related data from Last.fm sorted (descending) by Listeners.

I find it fascinating when you think numbers seem to say one thing but beg a question that’s answered with further investigation. Often, statistics are delivered in the simplest way, out of context, and presented in a way to support what the presenter wants to say, not what the data says.

For example, in the last election cycle, a list of questions was being passed around from one group of pollsters to the other. I know this because the first time I heard them, I complained that they were poorly written and their data would be meaningless as a result. Several weeks later, another group called me and when I told the young lady I’d like to re-state what I told the earlier pollster, she said it was impossible I’d heard them before because they’d just started calling people that day. I have to paraphrase from memory and, because I’m naturally honest, I won’t be able to recall it as horrible as it actually was … it was something like, “Do you think Obamacare should be repealed?” or “What are your feelings about Obamacare?” … both with equally bad Likert Scale answers. I told the pollsters it was obvious their poll was written so that no matter who they called, their “data” would say most people hated the Affordable Care Act (also notice they called it “Obamacare”). The truth is nobody likes it but there’s no way whoever presented the data was going to mention how many people thought it was “not enough” instead of “too much” — and they didn’t bother asking.

Anyway … on the other hand, sometimes data is good but could be great in the hands of a super genius data scientist like myself.

I collect data from the last.fm API — Listeners and Playcounts for artists, albums, and tracks. While the data is yummy straight from the oven, there’s a lot of mixing and kneading I can do before baking it that makes it yummier.

Figure 01 shows Radiohead has over 7 million more followers than the artist in second-place, Queen. The difference between 2nd and 3rd place (The Cure) is even bigger with approximately 11 million less. First of all, I can’t believe Radiohead is so popular. Second of all, when I first saw their number of listeners, I immediately asked myself if that resulted in an equivalent landslide of plays.

Maybe it’s cool to like Radiohead so they have a lot of listeners, but nobody actually listens to their music. You know, the guy with vinyl editions of their albums on his wall but if he heard a Radiohead song on the, you know, radio, he wouldn’t recognize it because he’s never really listened to them.

playcountDESCmovement.png
Figure 02: My artist-related data sorted (descending) by Playcount.

The top six and the bottom six stayed exactly the same as did Meat Loaf and the Amboy Dukes. The rest didn’t change much but the big winner in the set of artists I track was Saxon, gaining 4 slots (which, because these artists are merely the union of bands I’m curious about and the Rock and Roll Hall of Fame Class of 2019 still doesn’t tell you much beyond, “Huh, that’s interesting” if even that).

But wait, there’s more.

Playcount might be equally meaningless if — and this is the outcome I was hoping for — the listeners and playcounts were really close meaning a half-billion people tried the  casserole but nobody went back for seconds.

I can already tell that Radiohead fans listen to a motherload of Radiohead music because, as it turns out, their playcount is almost a half-billion. So I wrote a little Python script to find the ratios for me which I could then use for sorting.

ratioDescMovement.png
Figure 03: My artist-related data sorted (desc) by Ratio.

The top five still hold their places. Ozzy drops two places but that’s nowhere near as interesting as Saxon moving up 12 slots, Anvil … freaking Anvil … moving up 11 … Journey losing ten places and … and … Ronnie & the Prophets gaining nine because who can get enough of Dio singing doo-wop?

Let’s talk about Evil Stig (who gained +11) because I have some algorithms-in-progress to take care of this–and other–artists.

listenersCombinedHighlighted.png
Figure 04: Merging numbers for “the same” artists

Once the app is all done, it’ll auto-magically put artists together like Dio, Ronnie & the Prophets, Ronnie & the Redcaps, Heaven & Hell, Elf, and the Electric Elves because Ronnie James Dio (actually, there are some songs for that artist specifically as well) is in all of them. Why not also add Black Sabbath and Rainbow you ask? Because he was only in those bands for some of their albums. The algorithm(s) part of the solution is relatively easy compared to the tedious, manual table-building and table-populating part. Ugh.

RJD (blue overlay in Figure 04) would move up at least a couple slots with his 6-8 bands and Joan Jett would move up at least six with her three.

Advertisements

Motley Crue and Javascript D3

A few months ago, I posted a line graph showing the drastic increase in popularity of Queen and their song “Bohemian Rhapsody on Spotify. Well, Motley Crue has a movie out, too.

crueCurrentPopEtc.png
Figure 01: The Dirt was released on Netflix March 22

That’s just their Spotify Popularity score which, under most circumstances, is pretty useless. When the popularity score makes a significant change outside of the normal Spotify popularity ebb and flow (more on that later), it can make graphs that rock like the one above.

Before we get to their Spotify Followers and some techie stuff, here’s how the movie affected their albums’ popularity.

albumsMonth.png
Figure 02: Between March 17 and April 17, their albums also jumped up quite a bit.

The above was made with a little help from Photoshop. I don’t yet know how to make split columns like that with D3.js but the data is just as real.

crueFormattedFollowers.png
Figure 03: I started tracking Motley Crue‘s followers March 17, five days before Dirt‘s release.

Creating the above graph for Spotify Followers was more of a pain in the butt than I thought it would be. I thought all I’d have to do was replace my pop variable with followers. They’re even in the same table now, so … you know, right? I wanted to have multiple artists represented with multiple lines showing just how drastic that steep line is. But the range of popularity and followers for the artists I track in my database is HUGE so, even after narrowing down the artists, the Y-Axis went from  1 million to 30 million and the lines were all flat — or, at least, that’s how they looked unless I were to make the graph ten feet tall like this quickly made column graph showing Followers for this year’s Rock and Roll Hall of Fame nominees that’s ugly for all kinds of reasons. I totally thought I was doing something wrong.

friends.png
Figure 04: Lame, lame, lame.

Even just sorting the followers column and grabbing the artists above and below Motley Crue didn’t really help.

phpMyAdminPopFollow.png
Figure 05: I wanted to include artists in recent news as a point of reference but Queen and the Stones (let alone Eminem) blow the curve.

I have to say I fell in love with this little query just now:

SELECT p.pop, p.followers, a.artistName
FROM popArtists p
JOIN artists a ON a.artistID = p.artistID
WHERE date = '2019-04-17'
ORDER BY followers DESC

It was so much easier to write than the query I wrote while relatively inexperienced to get the most recent data. Anyway, my array of artists got so small and still looked so lame I gave up on that idea but I still had this problem to solve:

crueUnformattedFollowers.png
Figure 06: You’re looking to the left at the Y-Axis, that’s what’s different.

At first, I thought that would be easy. Just like I thought the whole thing would be. The “Motley Crue and Friends” multi-line chart was based on a more recent project. I learned how to make multi-line charts a while after I made the single-line version and, by that time, I’d also learned to stop doing this:

const w = 1100;
const h = 400;
const padding = 40;

and start doing this:

var w = 1000;
var h = 400;
padding = {
top: 50,
right: 30,
bottom: 350,
left: 100
};

The latter chunk of code makes everything delightful. Specifically, I could add some padding to just the left side, easily making room for those huge numbers in the tens of millions for “Motley Crue and Friends” because that file used a padding object with four properties instead of a single padding variable. When I increased the padding for what I guess I’ll call “Motley Crue All by themselves” it squished the graph from every direction making it even more hideous than it is up there if you can imagine that. Don’t. Really, don’t imagine that.

Originally, I had this humble bit o’ code:

const yAxis = d3.axisLeft()
.scale(yScale);

… which was fine for a simple 0-100 scale. To go from the horrifying Y-Axis in Figure 06 to the downright sexy Y-Axis in Figure 03, I learned to do this:

formatMillions = d3.format(".3s");
const yAxis = d3.axisLeft ()
.scale (yScale)
.tickFormat (function(d) { return formatMillions(d) } );

First, I looked through the epic-length documentation for D3-format (from a link I found in StackOverflow) which was not easy reading let me just tell you. It boils down to this:

  1. Make a function using format which is a … library? … but I didn’t have to import it or anything. I think that’s because I link to the D3cdn? … at d3js.org. I like naming my variables and functions something descriptive like “formatMillions.”
  2. Then, use that function on your data inside a tickFormat function after the yScale you already have.

That was easily the worst job I have ever done explaining anything.

There seemed to be an endless list of other, far more complicated ways to do this but I found “.2s” (the winning answer at SO) to be pretty simple. It rounds your millions down to one decimal and adds an “M” to the end. I didn’t want one decimal place, I wanted two — because otherwise all my ticks look like they have the same value! Whoever asked the question followed up with “How do I get two decimal places?” but nobody answered. Several places on SO and elsewhere all mentioned “.2s” and gave other complicated ways to get two decimal places with things I didn’t want like the symbol for euros and emojis and whatnot. I kept looking in this pit of documentation and despair for a while before I finally just tried “.3s” and it worked.

Rock and Roll Hall of Fame Class of 2019 Nominee: Kraftwerk

Part 3 in a series on Spotify and LastFM Data for the Rock and Roll Hall of Fame Class of 2019 inductees. In Part 2, I discussed my luke-warm at-best feelings about Roxy Music. Part 1 has charts comparing all of the nominees and inductees.

Kraftwerk, on the other hand, I totally get and not merely because I love their music. Nobody would or could deny their influence on countless artists and music in general. They, you know, created (or, at least, helped create) a whole genre.

Personally, I think influencers (not the Instagram kind) should be inducted before the bajillions of artists they influenced.

KraftwerkTop.png
I’m not yet collecting data on them from Last.fm. See Part 1 for an explanation of the chart.

As of this morning, they even edge out Gary Numan, arguably the king, if not also creator, of electronica, by a single popularity point on Spotify.

kraftwerkElectronic.png
Gary Numan, Kraftwerk, Ladytron, New Order

Wow, I just checked and Kraftwerk has twice as many Spotify followers as Gary Numan!

The popularity score for their albums on Spotify as of this morning:

KraftwerkAlbums.png
L – R: Autobahn, Radio-Activity, Trans Europe Express, The Man Machine, Computer World, Techno Pop, The Mix, Tour De France, Minimum-Maximum, 3-D The Catalog, 3-D The Catalog

I am utterly and completely ashamed that the only album I’ve ever owned by Kraftwerk is a greatest hits album, The Mix. The copy I have has a much cooler cover than the one in the chart above. Not only that, but their most popular song on Spotify is one I’ve never even heard of — “The Model” from The Man Machine.

Jesus Rocks Me, This I Know

Last night, I made a 200-song playlist of the greatest (read: my favorite) songs from the Golden Age of Christian Music (late 80s – early 90s).

https://open.spotify.com/playlist/04eEfPM3HnttETctCU0hNi

You may need to swipe or something to accept a setting or something if you’re using a mobile device.

jesusRocksMe.png

This most awesome playlist is guaranteed to strengthen, edify, and encourage you whether you feel like a rockstar or if, in the words of Sensei Lawrence, life just took big steaming dump in your mouth. Behold the artists I’ve included so far:

  • 12th Tribe
  • Adam Again
  • Alice Cooper
  • Allies
  • Amy Grant
  • Barren Cross
  • Bloodgood
  • Bob Dylan
  • Bride
  • Chagall Guevara
  • Crashdog
  • Daniel Band
  • Darrell Mansfield
  • David Zaffiro
  • DC Talk
  • Deitiphobia
  • Disciples of Christ
  • Don Francisco
  • Dynamic Twins
  • Freedom of Soul
  • Glenn Kaiser
  • Jacob’s Trouble
  • Jars of Clay
  • Jerusalem
  • Johnny Cash
  • Kaiser Mansfield
  • King’s X
  • L.S. Underground
  • L.S.U.
  • Larry Norman
  • Lifehouse
  • Lost Dogs
  • Lust Control
  • Mark Farner
  • mewithoutYou
  • Michael Knott
  • Michael Roe
  • Mike Knott
  • One Bad Pig
  • P.I.D.
  • P.O.D.
  • Petra
  • Prince
  • Resurrection Band
  • Saint
  • Scaterd Few
  • SFC
  • Steve Taylor
  • Stevie & the Saints
  • Stryper
  • Tim Cappello
  • Tourniquet
  • Trytan
  • Undercover
  • Vengeance Rising
  • XL and Death Before Dishonor

 

#CobraKai

#CobraKaiNeverDies

Curiosity Is Killing Me Over Trends In Spotify Data

A couple days ago, I noticed a downward trend in two random artists’ popularity. I instantly wondered if it was everybody (except the Latin artists) so, today, I ran some “tests.”

compareTimeFeb21.png

Yep, it’s definitely happening to everybody. Just to make sure, I did several groups.

allRappers.png
Note all the Christian rappers clustered down at the bottom.  Sad. They deserve better.

Seriously, isn’t that weird?

detroit.png

Doesn’t that bother you and gnaw at your brain?

classicElders.png

It’s just ridiculous. Most of the data from Spotify is worthless unless you look at Queen vis a vis the Bohemian Rhapsody film.

xianPunk.png
From September to present, Christian punk and christian alternative are … something.

I really need to make the text wrap in those legend items.

And I definitely want to go back to some of those and check out which artists have the more interesting … “lines” … there’s gotta be an actual term I should be using.

So I definitely need to create some buttons and menus so I don’t have to keep going into the files and manually typing which arrays I want the D3 to use.

As I predicted, the Latin artists weren’t affected at all.

latin.png

It vexes me. I’m very vexed.

Some individual artists have what I consider pretty interesting line-things. Those artists who, apart from the overall similarity of all the artists as shown in the preceding graphs sometimes jump around on their own (like in the preceding graphs — those that are doing their own thing that I’ll investigate further … later).

So, ignoring the universal (except for Alternativo and Rock en Espanol) late-July/early-August dip and the late-February dip …

rainbow.png

Rainbow keeps wanting to go up but goes back down almost immediately. Speaking of, I found a really great Rainbow concert on YouTube. Here is a playlist of above average videos of most of the concert and “Catch the Rainbow Part 1” which isn’t in the playlist. You’re welcome. I coded a bunch of the Python & LastFM stuff while watching it.

King’s X fans also can’t seem to make up their mind.

kingsX.png

And, lastly, Deep Purple

deepPurple.png

If you’re trying to compare those, be aware of the X-Axis — they show different spans of time. When looking at an individual artist’s page, the graph begins and ends with whenever I added them to my database (begins) and ends with present day so you could be looking at a span of just a few weeks or a span of well over one year.

First Tiny Bits o’ Data from LastFM

I can’t even begin to tell you how much fun I’m having with this mixture of Python, the MusicBrainz API, and the LastFM API. Much more to come about the process itself but, for now, just know that the number of Alice Cooper listeners rose from 1,365,754 on Feb 14, 2019 to 1,365,860 on Feb 15 for a total increase of 106. His total playcount rose from 31,123,429 to 31,126,727 — an increase of 3,298 plays. Lovely data viz will follow the procedural tutorial.

Streamlining PopRock UI

The navigation has mostly been for me as I test the app so, sometimes, the top menu is two full rows of buttons so I can easily move around. The goal, however, is to make everything dynamic and intuitive. I’ve just started making that happen.

index.png

There’s no longer a menu to choose an artist from — the user goes to the main artist list (above, now the index.php page instead of being called allArtistsList.php or whatever it was) and clicks the artist’s name (I still need to make the images clickable) to open an artist page (combining the former list of popularity scores and D3 line chart representing those scores) that now includes a column chart of that artist’s albums’ popularity scores.

QueenArtistPage.png
Queen is plateauing despite wins from the Golden Globes and noms from the Academy Awards.

I still don’t know quite what I’ll do with those long rows of albums.

All these links are now based on URL parameters instead of cookies. Much, much easier.

On the albums list page, each album is a link to that album’s page with popularity scores for that album’s tracks. I want to replace that by making the album art on the artist page links to those album pages … is this getting confusing?

Removing the need for all these pages in the top nav makes room for other items like comparing artists and such.

compareTime.png
Comparing artists’ popularity over time. User clicks artist at bottom to toggle visibility in graph.

The goal for the line graph above and column chart below is two-fold:

  • At present, the genres page is just a list of all my artists and the genres Spotify assigns them. Soon, the user will be able to click the genre of an artist they like and it will open a page displaying a column or line chart of all artists in that genre.
  • Custom charts with drag & drop capability so user can choose artists to compare
compareCurrent.png
Right now, this uses a static array of Rock en Espanol artists.

Probably my simplest near-future task is making some buttons for the few “groups” of related artists I have for the following:

steveTaylor.png
This static array displays albums by all four groups Steve Taylor is in.