Spotify and LastFM Data for Rock and Roll Hall of Fame Class of 2019

The ongoing project I’m usually busiest with is PopRock — a web app that tracks statistics and creates data visualizations using data from the Spotify Web API and the Last.fm API.

From Spotify, I grab artist Popularity and Followers (daily) as well as Popularity for albums and tracks (weekly). That is

  • 327 artists
  • Their 5,742 albums
  • The 75,283 tracks from those albums

If you’ve never heard of Last.fm, it gathers data via “scrobbles” from all participants who have a “scrobbling” app installed. When you, the music lover, listen to a song using iTunes, Spotify, etc. the app sends that bit of information as a “scrobble” to Last.fm. As often as I remember to, I run a Python script that gathers the Listeners and Playcount for each artist, their albums and tracks. At the moment, this task is so cumbersome, I keep the list relatively short — 32 artists.

What brings joy to my nerdy little heart is when there’s an event that creates an anomaly or jump in my data. For example, a movie about Queen causes a spike in popularity — especially for a particular song (“Bohemian Rhapsody”) or a movie about Motley Crue causes a spike in their popularity.

What brings sadness to my heart is how little influence the Rock and Roll Hall of Fame has on my little data exercises. For the most part, neither inductions nor nominations cause much of a change. Having said that, I look forward to the announcement of nominations every year as well as to the broadcast of the induction ceremony. The ceremony was held last week and the ceremony will air on HBO April 27. To celebrate that, I’m going to share some data and charts for this year’s nominees and inductees.

Below (Figure 01) is the Spotify popularity for the Class of 2019.

compareBars.png
Figure 01: L-to-R Def Leppard, Janet Jackson, Radiohead, Roxy Music, Stevie Nicks, the Cure, the Zombies

I’ve already written about how Spotify calculates their popularity score numerous times and every article about the RRHoF repeats their rules and controversy so forgive me if I don’t regurgitate all of that here.

Induction Process and Eligibility Requirements, etc.

compareLines_Inductees.png
Figure 02: A line starts on the day (or day after) I add an artist to my database.

As you can see, all artists flow together for the most part — because Spotify’s popularity scores are relative. That’s what caused all of them to go down together in late June, climb back up and fall again together in February. Given that, the lines, in a vacuum, would be pretty straight. Knowing that, the nominations announced in December 2018 seem to have no effect.

Speaking of nominations, here are those who were nominated last year but not inducted.

compareBars_Noms.png
Figure 03: L-to-R: Devo, John Prine, Kraftwerk, LL Cool J, MC5, Rage Against the Machine, Rufus featuring Chaka Khan, Todd Rundgren

Like many people, I find it extremely difficult to keep my thoughts about this to myself. At least three of those artists deserve to be in more than at least three of the inductees. But, of course, if the RRHoF used reason and logic, people wouldn’t write motherloads of articles and blog posts about them twice a year.

compareLines_Noms.png
Figure 04: Yes, I know I need to tweak a couple things. It’s a work in progress.

The only artist I would say was really affected by their nomination is the MC5. I would expect all of them would bump up because the public was reminded of these artists and listened to them like, “Oh, yeah, I haven’t listened to LL Cool J in forever” but, in the case of the MC5, I think it’s more like, “Who are the MC5?” and I am just fine with a new generation being exposed to them by any means necessary. Very similar to another highly-influential Detroit band, The Stooges (inducted 2010). I’m still waiting for Ted Nugent to get selected for consideration to be put on the ballot. Once.

Because popularity is relative, you might be thinking the line charts are pretty useless and you’d be right. If I look at each artist individually, however,  and change the Y-axis from a 0-to-100 scale to a lowest-to-highest scale, they become (a little) more useful. Sometimes. This begins a series of posts about each inductee that do exactly that with some more useful statistics and data visualizations.

Before we go on, however, here’s something more useful — a graph using each artist’s followers on Spotify.

followerColumnsX.png
Figure 05: L-to-R: Def Leppard, Kraftwerk, Janet JacksonJohn Prine, DevoLL Cool J, MC5, RadioheadRage Against the Machine, Roxy MusicRufus featuring Chaka Khan, Stevie Nicks, The Cure, The ZombiesTodd Rundgren

Day-um! Radiohead has some fans, yo!

That wide range between, for example, The Zombies (who deserve better, by the way) and Radiohead makes for what could be considered a clunky chart.

In other charts, the ratio for popularity (which is, conveniently, on a scale of 0-100) is 1:2. One popularity point gets represented by two pixels so I can fight against the opposite problem of a chart that’s too small. The ratio in the above chart is 5000:1. 5000 followers to one pixel.

Chrome’s dev tools were invaluable in troubleshooting that.

I love how “good” I’m getting at math now that it’s always in a real, practical context. For the labels on top of the columns I first checked to see whether the number was above or below one million.

  • If it was below, I divided by 1000 and used Javascript‘s toFixed(0) method to strip away the decimal places — which super conveniently converts the integer to a string which I concatenated with a “k”.
  • If it was above, I divided by one million, used toFixed(1) and added the “M”.
Advertisements

Curiosity Is Killing Me Over Trends In Spotify Data

A couple days ago, I noticed a downward trend in two random artists’ popularity. I instantly wondered if it was everybody (except the Latin artists) so, today, I ran some “tests.”

compareTimeFeb21.png

Yep, it’s definitely happening to everybody. Just to make sure, I did several groups.

allRappers.png
Note all the Christian rappers clustered down at the bottom.  Sad. They deserve better.

Seriously, isn’t that weird?

detroit.png

Doesn’t that bother you and gnaw at your brain?

classicElders.png

It’s just ridiculous. Most of the data from Spotify is worthless unless you look at Queen vis a vis the Bohemian Rhapsody film.

xianPunk.png
From September to present, Christian punk and christian alternative are … something.

I really need to make the text wrap in those legend items.

And I definitely want to go back to some of those and check out which artists have the more interesting … “lines” … there’s gotta be an actual term I should be using.

So I definitely need to create some buttons and menus so I don’t have to keep going into the files and manually typing which arrays I want the D3 to use.

As I predicted, the Latin artists weren’t affected at all.

latin.png

It vexes me. I’m very vexed.

Some individual artists have what I consider pretty interesting line-things. Those artists who, apart from the overall similarity of all the artists as shown in the preceding graphs sometimes jump around on their own (like in the preceding graphs — those that are doing their own thing that I’ll investigate further … later).

So, ignoring the universal (except for Alternativo and Rock en Espanol) late-July/early-August dip and the late-February dip …

rainbow.png

Rainbow keeps wanting to go up but goes back down almost immediately. Speaking of, I found a really great Rainbow concert on YouTube. Here is a playlist of above average videos of most of the concert and “Catch the Rainbow Part 1” which isn’t in the playlist. You’re welcome. I coded a bunch of the Python & LastFM stuff while watching it.

King’s X fans also can’t seem to make up their mind.

kingsX.png

And, lastly, Deep Purple

deepPurple.png

If you’re trying to compare those, be aware of the X-Axis — they show different spans of time. When looking at an individual artist’s page, the graph begins and ends with whenever I added them to my database (begins) and ends with present day so you could be looking at a span of just a few weeks or a span of well over one year.

Streamlining PopRock UI

The navigation has mostly been for me as I test the app so, sometimes, the top menu is two full rows of buttons so I can easily move around. The goal, however, is to make everything dynamic and intuitive. I’ve just started making that happen.

index.png

There’s no longer a menu to choose an artist from — the user goes to the main artist list (above, now the index.php page instead of being called allArtistsList.php or whatever it was) and clicks the artist’s name (I still need to make the images clickable) to open an artist page (combining the former list of popularity scores and D3 line chart representing those scores) that now includes a column chart of that artist’s albums’ popularity scores.

QueenArtistPage.png
Queen is plateauing despite wins from the Golden Globes and noms from the Academy Awards.

I still don’t know quite what I’ll do with those long rows of albums.

All these links are now based on URL parameters instead of cookies. Much, much easier.

On the albums list page, each album is a link to that album’s page with popularity scores for that album’s tracks. I want to replace that by making the album art on the artist page links to those album pages … is this getting confusing?

Removing the need for all these pages in the top nav makes room for other items like comparing artists and such.

compareTime.png
Comparing artists’ popularity over time. User clicks artist at bottom to toggle visibility in graph.

The goal for the line graph above and column chart below is two-fold:

  • At present, the genres page is just a list of all my artists and the genres Spotify assigns them. Soon, the user will be able to click the genre of an artist they like and it will open a page displaying a column or line chart of all artists in that genre.
  • Custom charts with drag & drop capability so user can choose artists to compare
compareCurrent.png
Right now, this uses a static array of Rock en Espanol artists.

Probably my simplest near-future task is making some buttons for the few “groups” of related artists I have for the following:

steveTaylor.png
This static array displays albums by all four groups Steve Taylor is in.

PrezPlayPro Progress

I’ve made so much progress since I last revised my schema in Nov 2017. Most of that progress has been in the last week — two weeks at the very most. The Mobile Web Specialist nanodegree program I’m in as a Google Udacity Scholar via a Grow With Google scholarship has made my skillset and confidence go all supernova.

See also: I have a dream come true (2012 results made manually in Photoshop)

Here is what I’m up to:

pppProgress.png
Gray states are those with no socialist candidates (or there were socialist candidates but they received zero votes).

This query fetches only socialist candidates (excluding Hillary Clinton in one state where she’s actually on the ballot as a socialist — she’s not included because then she’d win all the states … until I fix my query by, hopefully, finally completing the affiliations table below) to see who among would “win” among the socialist candidates.

Here’s the new schema … I still haven’t added a stateAbbr column to the affiliations2016 table which, I think, is the reason for a problem I’m having with my first major query.

schema_061918.png

I need some buttons to change what results are fetched and displayed including but not limited to:

  • Results if all the “left-ish” third parties voted for a single candidate — would that candidate have beaten Trump?
  • If all the right-ish votes went to Trump, would he have have won the popular vote?

What is Instructional Design, really?

7 Job Requirements for Real Instructional Designers

Real Instructional Designers are teachers. We take knowledge from Subject Matter Experts who don’t have the skills, time, or desire to pass along their expertise effectively. Just because you can use Storyline, Camtasia, or Captivate doesn’t mean you’re an Instructional Designer–even the most anti-social or misanthropic SME can use Powerpoint (and those tools are mostly just overpriced Powerpoint). Grade-schoolers can use Powerpoint and master software more complex and advanced than Storyline faster than you. We intake complex and/or confusing material fast, dissect and analyze it, and output it in ways that are simply constructed and easily assimilated. And we can do that even without software.

Real Instructional Designers are editors. We identify what is essential, throw out the rest, and organize what remains in the order it makes the most sense to the student. Most of what an SME or project champion gives you is unnecessary at best.

Real Instructional Designers are detectives. What the client thinks they need might not be what they actually need. What the client thinks is the problem might not really be the problem. Maybe the end-user (the student) already has all that knowledge and doesn’t use it. The reason or motivation for not using existing knowledge is the problem and the solution. What they think is the only solution might not be the best solution. We help them ask the right questions and find the best answers.

Real Instructional Designers are researchers. Information provided by SMEs, project champions, and/or stakeholders might not be factually correct. We find the answers, not just regurgitate what’s given to us.

Real Instructional Designers are UX/UI designers. We present material in a way that is easy to use and understand. We instinctively know whether audio, images, video, text, simulations, or games are the best way to learn any given bit of knowledge. We find out how the end-users think and works so the students access and achieve everything required of them by following only their intuition. Students know what is expected of them, where they are in the training timeline, and how to navigate the learning process without being told via separate instructions or a tutorial.

Real Instructional Designers are Gamifiers. We make learning not only painless, not merely engaging, but fun. Players (end-users, students) know the objectives, how to play the game, and how to win (perform better, pass an assessment). We make learning active (not passive), challenging, and realistic (practical and relevant with real world examples and application).

Real Instructional Designers are Data Scientists and Data Visualizers. Students understand the purpose (why) of what they’re doing (not “you’ll learn to tie your shoes” objectives or key points) and the method (how) for achieving their objectives. We reshape dry, boring, complex, confusing information and make it not only palatable but appetizing and yummy–students should always want to come back for seconds. We provide concrete evidence for the validity and value of the time they’re investing. We evaluate the situation before designing and developing the material keeping the end in mind and evaluate the results which don’t end when the student takes a lame multiple-choice quiz. We provide the evidence of the need beforehand and the success afterward to the stakeholders in clear, dynamic, inspiring ways.

Make It Paint

makeItPaint

I really loved this exercise from Khan. It’s simple, the code isn’t glamorous, but I love the result and I want to play with it later.

I’m really getting excited about Processing.js, the … hmm … it’s not a library, apparently, is is a “port of Processing, a programming language designed to write visualizations, images, and interactive content. It allows web browsers to display animations, visual applications, games and other graphical rich content without the need for a Java applet or Flash plugin.” (Wikipedia)

Anyhew … I’m excited about using it for “data visualizations,” “information design,” etc. because I really love me some well-done and interesting infographics. I just love the idea of live, real-time, dynamic, graphics and animations.

This may be another piece to my ultimate (right now) project … I hope I learn everything I need to in time to do it … it’s rather time-sensitive.