Motley Crue and Javascript D3

A few months ago, I posted a line graph showing the drastic increase in popularity of Queen and their song “Bohemian Rhapsody on Spotify. Well, Motley Crue has a movie out, too.

Figure 01: The Dirt was released on Netflix March 22

That’s just their Spotify Popularity score which, under most circumstances, is pretty useless. When the popularity score makes a significant change outside of the normal Spotify popularity ebb and flow (more on that later), it can make graphs that rock like the one above.

Before we get to their Spotify Followers and some techie stuff, here’s how the movie affected their albums’ popularity.

Figure 02: Between March 17 and April 17, their albums also jumped up quite a bit.

The above was made with a little help from Photoshop. I don’t yet know how to make split columns like that with D3.js but the data is just as real.

Figure 03: I started tracking Motley Crue‘s followers March 17, five days before Dirt‘s release.

Creating the above graph for Spotify Followers was more of a pain in the butt than I thought it would be. I thought all I’d have to do was replace my pop variable with followers. They’re even in the same table now, so … you know, right? I wanted to have multiple artists represented with multiple lines showing just how drastic that steep line is. But the range of popularity and followers for the artists I track in my database is HUGE so, even after narrowing down the artists, the Y-Axis went from  1 million to 30 million and the lines were all flat — or, at least, that’s how they looked unless I were to make the graph ten feet tall like this quickly made column graph showing Followers for this year’s Rock and Roll Hall of Fame nominees that’s ugly for all kinds of reasons. I totally thought I was doing something wrong.

Figure 04: Lame, lame, lame.

Even just sorting the followers column and grabbing the artists above and below Motley Crue didn’t really help.

Figure 05: I wanted to include artists in recent news as a point of reference but Queen and the Stones (let alone Eminem) blow the curve.

I have to say I fell in love with this little query just now:

SELECT p.pop, p.followers, a.artistName
FROM popArtists p
JOIN artists a ON a.artistID = p.artistID
WHERE date = '2019-04-17'
ORDER BY followers DESC

It was so much easier to write than the query I wrote while relatively inexperienced to get the most recent data. Anyway, my array of artists got so small and still looked so lame I gave up on that idea but I still had this problem to solve:

Figure 06: You’re looking to the left at the Y-Axis, that’s what’s different.

At first, I thought that would be easy. Just like I thought the whole thing would be. The “Motley Crue and Friends” multi-line chart was based on a more recent project. I learned how to make multi-line charts a while after I made the single-line version and, by that time, I’d also learned to stop doing this:

const w = 1100;
const h = 400;
const padding = 40;

and start doing this:

var w = 1000;
var h = 400;
padding = {
top: 50,
right: 30,
bottom: 350,
left: 100

The latter chunk of code makes everything delightful. Specifically, I could add some padding to just the left side, easily making room for those huge numbers in the tens of millions for “Motley Crue and Friends” because that file used a padding object with four properties instead of a single padding variable. When I increased the padding for what I guess I’ll call “Motley Crue All by themselves” it squished the graph from every direction making it even more hideous than it is up there if you can imagine that. Don’t. Really, don’t imagine that.

Originally, I had this humble bit o’ code:

const yAxis = d3.axisLeft()

… which was fine for a simple 0-100 scale. To go from the horrifying Y-Axis in Figure 06 to the downright sexy Y-Axis in Figure 03, I learned to do this:

formatMillions = d3.format(".3s");
const yAxis = d3.axisLeft ()
.scale (yScale)
.tickFormat (function(d) { return formatMillions(d) } );

First, I looked through the epic-length documentation for D3-format (from a link I found in StackOverflow) which was not easy reading let me just tell you. It boils down to this:

  1. Make a function using format which is a … library? … but I didn’t have to import it or anything. I think that’s because I link to the D3cdn? … at I like naming my variables and functions something descriptive like “formatMillions.”
  2. Then, use that function on your data inside a tickFormat function after the yScale you already have.

That was easily the worst job I have ever done explaining anything.

There seemed to be an endless list of other, far more complicated ways to do this but I found “.2s” (the winning answer at SO) to be pretty simple. It rounds your millions down to one decimal and adds an “M” to the end. I didn’t want one decimal place, I wanted two — because otherwise all my ticks look like they have the same value! Whoever asked the question followed up with “How do I get two decimal places?” but nobody answered. Several places on SO and elsewhere all mentioned “.2s” and gave other complicated ways to get two decimal places with things I didn’t want like the symbol for euros and emojis and whatnot. I kept looking in this pit of documentation and despair for a while before I finally just tried “.3s” and it worked.


Christian Metal Bar Graph

My latest “feature” is a bar graph to compare artists’ popularity. Right now, I have separate arrays based on genres and do this manually. I want the user to be able to choose a genre which, at this point, would change the array used by the MySQL query in PHP. I’m having trouble with that, however.

The final goal would be to have a grid of artist images and the user drags and drops them onto the chart area to add them — either in the line chart shown in a previous post or like the one below.

In real life, you can hover over the artist’s image to see the artist’s name. Since you can’t see that in this screenshot, the list of artists (L-R) is beneath the screenshot. I changed it to a numbered list from a bulleted list for easy counting.

Screen Shot 2018-12-22 at 11.58.41 AM.png
What’s especially harsh for Christian artists is so many of them have incomplete discographies on Spotify. Not Spotify’s fault — Christian record companies that released good music during the 80s and 90s disappeared almost as quickly as they started. Metal fares much better than, say, rap.

Sorry — I don’t know why WordPress is so inconsistent about whether it lets you click images to see them full size and/or allows me to place them in such a way that it does that. What you’re able to do when creating and editing posts seems to change with each browser and operating system and I can’t keep track.

  1. Barren Cross
  2. Believer
  3. Bloodgood
  4. Bride
  5. Deliverance
  6. Gold, Frankincense, & Myrrh
  7. Jerusalem
  8. King’s X
  9. Living Sacrifice
  10. Mortification
  11. P.O.D.
  12. Resurrection Band
  13. Saint
  14. Seventh Angel
  15. Stryper
  16. Tourniquet
  17. Trytan
  18. Vengeance Rising
  19. Veni Domine
  20. Whitecross
  21. XL and Death Before Dishonor

For mobile users, I need to make the artists display vertically so the bars go horizontally. These rows of columns (does that make sense?) get really wide really quick which you can see when viewing albums for artists with huge discographies as shown in the screenshot below.

Screen Shot 2018-12-22 at 12.14.27 PM.png

Even with my default width now set to 2400px, David Bowie runs out of room as do artists like the Rolling Stones.

Do I Need to Be Good At Math to Learn Programming?

Frequently asked question. My immediate response is no. I’m sure it shortens the learning curve, however.

The good news is, if you suck at math, learning to program can change that which then does help you become a better programmer.

That’s certainly been the case with me. In high school, I sank from Algebra to Basic Algebra to Remedial Math and then kinda failed out of just about everything my sophomore year. I hated math. Why? Because math class sucked. Why? Because the struggle seemed to have neither end nor hope. As an adult, Star Trek and other fun stuff made me a big astronomy and space geek. I wanted to learn about space travel. It took a single question (and answer) for geometry and physics to have a context in which I could understand them so they made sense. I’m no Stephen Hawking, but trading fear for joy is a big deal to me.

What’s unfortunate for millions and millions of students for decades and decades is their question, “When am I ever going to use this?” is met with blank stares.

Using D3 to create data visualizations has helped me learn algebra. Again, having a practical use for something makes it much easier to learn.

My two recent project challenges and “learning issues” were (please forgive and ignore the lingo):

  • Understanding how to create “padding” or “margins” around an SVG–the black area in the above image containing the above graph, title, and legend (I put those in quotes because they are, in this context, figures of speech and not the padding or margins you may be used to using in HTML and CSS) especially when it came time to use those numbers with scales–the rules for telling data how to accurately display in given dimensions (if that’s gobbledygook, don’t worry about it — just know it was already difficult before the Masters of D3 I was learning from explained it like it was child’s play).
  • Getting the above twenty metal bands to display neatly and in rows. As it turns out, an SVG isn’t like a DIV or TD which can force contents to wrap.

What Made the Math Easy To Learn

I had a practical application. Something I could picture. A context for the numbers and equations in the algorithm (algorithm: a series of steps to solve a problem or complete a task — holy cow! doesn’t sound so mysterious or difficult now, does it?) instead of a bunch of numbers and stuff that I was expected to blindly memorize.

Math is a language. Imagine you’re in Cooking Class and the teacher speaks a foreign language and you’re not allowed to see the ingredients or the process. Then the teacher tells you you’re stupid because you don’t get it. So you get sent to Basic Cooking which is no easier. In fact, it’s worse because none of the other students really want to be there and they feel crummy, too. So you get sent down to Remedial Food which is even worse because the teacher hates their job and the students are all hoodlums.

It doesn’t have to be that way. The teacher could provide examples, real-life application, and show you the utensils, kitchenware, ingredients, and explain why you’re doing what you’re doing.

When you’re programming, you know why you need to know something. You know what you don’t know. Fortunately, if you’re building some type of User Interface or data visualization, you can see the results of your mistakes — see how that thing is too big or too far to the left or the pie is all burned on top? — so mistakes are often learning treasures.

Full Disclosure: So you don’t get discouraged or needlessly frustrated, know that sometimes the best of us can’t figure out why something doesn’t work. For hours. Then someone walking by will say, “you know you forgot a semicolon there, right?” and you wasted hours without learning anything other than, yes, you’re an idiot sometimes. We are all idiots sometimes–but not as much as you might think you are.

Full Disclosure Part 2: Sometimes you can’t figure out why something does work. You can waste just as many hours on that, too. Don’t. Move on. Go learn something else. Or go outside for recess and play with others. You need that.

Approximately one year of popularity data for Mortification, Deliverance, Overkill, Living Sacrifice, Cannibal Corpse*, Pantera, Slayer, Megadeth, Testament, Metallica, Kreator, Gwar, Anthrax, Vengeance Rising, Seventh Angel, Celtic Frost, Venom, Death Angel, Sodom, and Exodus.



Prez Play Pro Progress

Thus far, if I spend this much time fixing stuff, it’s really frustrating because I’m not learning anything, I’m just trying to figure out what’s broken and why — but it wasn’t my own mistake or lack of knowledge.

This time, however, it is TOTALLY my lack of knowledge so each time I hid a bump, I learn something. Each problem I solve — I learn how and why what I did wasn’t working and how to do it correctly!

A lot of my work super-recently has been learning and using Javascript‘s map, reduce, and filter methods. I was depending on SQL queries (my database being MySQL, if you’re curious … though I was using MariaDB before changing hosting plans) to get the exact results and data visualization I wanted but, for multiple reasons, I’d prefer getting all the results from my database and manipulate the data client-side with Javascript.

Much of what follows is actually from an email I just sent to the American Nazi Party replying to an answer they sent to a question I asked a couple weeks ago. Say what you want about nazis, whenever I’ve asked a question, they’ve sent a polite and meaningful answer. Not all candidates/parties are responsive, let alone courteous. Anyway, I was explaining why I asked my questions and what I was doing.

So, here are my latest screenshots along with what is still in progress, etc.

If only what I consider “right” candidates run — excluding Trump because, of course, he’d win every state, then Gary Johnson wins every state. If we exclude him as well, Darrell Castle (usually Constitution Party) wins all states in which he runs except Arkansas where Jim Hedges wins. The gray states have no conservative/right candidates.

Screen Shot 2018-11-17 at 10.17.24 AM.png

Whatever happened to the Natural Law Party and Reform Party? Maybe the former are all write-in candidates (Darn it! I need to get write-in data!). I know the Reform Party was only on a couple state ballots in the 2012 election — not only that but with two different candidates. I read somewhere at least one state Reform Party nominated a different candidate because, despite how conservative Andre Barnett was (and still is, I assume), they didn’t like that he was (and still is, I suppose) black.

If only “left” or even just socialist candidates run, Jill Stein wins every state except Nevada, South Dakota, and Oklahoma (off the top of my head I think it’s simply because no “left” candidates besides Clinton were on the ballot there). If we only include socialists except Jill Stein, these are the results:

Screen Shot 2018-11-17 at 10.24.07 AM.png

Michael Maturen looks like he wins the most states (I don’t have electoral college votes in the code yet) and there are five tied in second place for number of states. Again, gray means there are either no socialist candidates in those states (except Stein) or, if there are — I need to clean up my data to make sure — there are candidates but none received any votes (which I find difficult to believe).

Once I get these all working, I’ll start adding previous elections and, hopefully, the next one won’t take two years to put together (in my own defense, most of that two years was the learning curve).

Refactoring Artist Management in MyRockinApp Part 1

Adding artists has been an ongoing pain in my ass during the entire lifetime of this project for many reasons. All of those reasons have one root reason — when I want to get something done, I tell myself I don’t have time to learn how to do it better … and I spent all that time learning how to do it the way it is now … I want to keep practicing that …

TL;DR — I am taking the time to completely refactor (as well as redesign) the entire app so it is more efficient, elegant, and easier to use.

The horrifying current state of affairs

I add artistIDs to a new array. Usually, they’re artists I have yet to add, artists I’ve realized I didn’t have, or there’s a new class of Rock and Roll Hall of Fame nominees.


Then use the Spotify WebAPI to get their data and insert rows into the artists table of my database. I do this the most stupid way possible — by changing the value of a variable in a PHP file and then going to that URL in a browser.

Then, I … don’t judge me! … I’ve been manually adding lines like this …

Yes, I know my camelCase wasn’t consistent. That’s why I’m doing and writing this.

So that … um … I can … well … manually add them to the “Choose Artist” menu like this:


Needless to say, I’ve often remembered to update one of those files and not the other. Or mismatch artistIDs and artistNames.

Because my albums table has a foreign key for artistID (from the artists table) and the tracks table has a foreign key for albumID from the albums table … sometimes forgetting to add things in order of artist, albums, tracks is another pain in the ass.

Also, I have separate files and functions for each little task and purpose — and not in the good way you’re probably picturing in your mind. They’re all redundant and overlap in countless ways.

I’ve know there was a better way for a long time but I’m always busy and always in a hurry. But I’m fixing that now. I’m forcing myself to. There’s a growing list of things like adding genres and grouping artists i.e. Dio, Ronnie James Dio, Elf, Rainbow, Black Sabbath, etc. that — to keep my sanity intact — require me to create some forms and functions resembling code written by a professional.

I’m doing this now for two reasons — I have the time and patience … but, more importantly, I learned a mother-butt-load earning Udacity’s Mobile Web Specialist nanodegree as well as the other resources I discovered while completing the program.

Now is the time on Sprockets where I replace much of the PHP in the app with Javascript. All of the Front End stuff, in fact, will now be strictly HTML, CSS, and Javascript. The PHP will be hidden in the back like Geoff Nicholls and any other heavy metal keyboardist.

Serendipity and Service Workers

Over the last few days, I’ve been updating an Acrobat JavaScript app (totally awesome dynamic PDF that may be the one thing I’m most proud of) I made for a client and realized I could add use Service Workers in Stakeout — the mobile Web App I started building around and on top of the PDF.

While working on the PDF, I went back to the web app to apply some of what I learned in the Grow With Google Mobile Web course that’s finishing up and happily realized it is a far better candidate to apply everything we learned about IndexedDB than the popRock (personal) project I’ve been trying to force it into as practice. And … AND … Service Workers which I couldn’t even come up with an impractical reason for forcing into popRock. But wait, there’s more …

Not only that but it’s a perfect candidate for learning and experimenting with pouchDB and couchDB — two IndexedDB alternatives a bunch of my fellow GoogleUdacityScholars keep suggesting because, if for no other reason, this app could absolutely completely benefit from being offline first!

The timing of this is super-awesome. For a couple reasons, I hadn’t worked on the web app in about a year and this gave me both the motivation and the tools to jump back on it.

Where Were These Resources Weeks Ago?

After learning about IndexedDB in the GrowWithGoogle Mobile Web course, I’m enhancing my big personal project to create a local backup of the MySQL database using IndexedDB with PHP and jQuery

If only I’d been aware of this “Google Search” thing a month or two ago … 😉

From MySQL to IndexedDB

Also mentions PouchDB and CouchDB

Synchronization between mysql and IndexedDB

SyncedDB repo

Client-side database solutions: IndexedDB (says “teh codez”)

Create Your Own To-Do App with HTML5 and IndexedDB

Offline Add, Edit, Delete Data in HTML5 IndexedDB