Overview

I worked on this project in 2 phases: Proof of Concept and Productionizing. Phase 1 includes creating the standalone visual, which took roughly 4 weeks. Phase 2 includes using Python flask to build the application, generalizing layout of the flower visual, and deploying, which took roughly another 4 weeks. I started Phase 2 6 months after completing Phase 1 because I learned about deploying data applications from the Data Engineering team at work. In that 6 month gap, I improved as a developer with the help of my coworkers and my series of personal projects that I felt ready to productionize my work.

Part 1: Creating the Proof of Concept

In my standalone visual, I knew I wanted to work with a song from the Electroswing genre. It’s jazzy, groovy, and more complex than mainstream pop songs. I chose Lone Digger by Caravan Palace because I was familiar enough with how the song goes to be able to work with it.

Data Collection

I began my design process by browsing the Spotify API to see what data was available. I used Jupyter Notebook to check out samples of data from the relevant endpoints, like artist, track, audio analysis. Everything was straightforward to me except for the chroma vector. The data structure looked so odd, and there was a lack of examples online. Wikipedia actually had the best explanation of the chroma feature because of the visual. This allowed me to replicate it using matplotlib.

In this process, I realized that music is an opportunistic space with interesting data and not much existing visualization. Most of the visuals I saw were research-based, and didn’t look aesthetic. It would be fun to make this chroma feature look pretty and fascinating at the same time.

Mockups

I started by drawing random objects on a piece of paper to visualize data with the hierarchal structure of sections, bars, and individual notes. I decided to go with flowers (inspired by film flowers by Shirley Wu) because I could split it into 3 layers (flower, petals, petal colors). I hard coded the positions based on the order on the x-axis and section pitches on the y-axis.

After building the flower skeleton, I used Photoshop in my planning stage to figure out how to paint the flowers using individual notes. Based on my Photoshop mindset, I used a blur brush and clipping mask, which I would replicate with code using the <defs> element.

Coding with D3

The biggest challenge I had was passing data from one level of the hierarchy to the next. Not having much experience working with JSON, I defaulted to exporting my data as a .csv flat file, but it made the link between flowers, petals (subset of flower), and notes (subset of petal) impossible. Doing other tasks like creating a clipping mask was difficult in combination with solving the hierarchal data structure. I redid my data format by converting it to JSON, and my data’s subset-building functions magically worked with the following usage of .data() in D3.

d3.selectAll("g")
	.data(flowerJSON)
	.enter()
	.append("g")
		.attr("x", d => d.x) // x-position of pistil
		.attr("y", d => d.y) // y-position of pistil
		.selectAll("path")
		.data(f => f.petals)
		.enter()
		.append("path")
			.attr("d", "M0,0 Z"); // insert path for drawing a single petal

Besides things not working the way I wanted for a long time, another mental drain was the visual not looking as good as I planned. I couldn’t make my results look like the one I had photoshopped because I was “using real data” (and needed to stay true to it). I experimented with different backgrounds, chroma thresholds, colors, and the feature that made it look GREAT ended up being the computationally expensive Gaussian blur. This visual loads pretty slow if I ran a simulation on it, so I loaded the filter after the simulation when I worked on Phase 2.

The following 2 images are my drafts that didn’t look good with the chroma vector. The first image doesn’t have enough blur and has a background that acts as additive light, so it’s hard to see the pitches if the flower has many petals. The second image has the blur I want but didn’t have the right background; the flowers look like they have holes.

Final Proof of Concept

We are done after a lot of editing!

I also added my interpretation notes when you hover over the button “hover for notes” on the bottom left.

Part 2: Productionizing as a Web Application

Refactoring Code

My main goal in refactoring was the following:

• Generalize my javcascript code to draw flowers for any song
  • Build a simulation to determine the positions of all flowers because I hard-coded the x and y positions for flowers that I created in my original work.
  • Turn my visual into a class for better organization.
• Turn my Jupyter Notebook of data wrangling into a python class that does the following:
  • Use the Spotify Search API to retrieve the track’s URI based on user input.
  • Use the Spotify Audio Analysis API to get the breakdown of the song.
  • Manipulate the data with pandas and return a JSON.
• Build a python Flask object that would feed the generated data from the server into the javascript, client-side script.
• Deploy the new web application using Heroku.

See my GitHub for the final result!

Using Python Flask

I used this YouTube tutorial by Corey Schafer to get started on using flask.

Since I already made a standalone version, I created a templates subdirectory, a default location for flask and replaced a lot of my index.html code with expressions enclosed in double brackets.

To send the data JSON over from the server (python web application) to the client (javascript), I added the data as a parameter in flask’s render_template function.

@app.route("/grow")
def grow():
	"""
	This code does stuff before returning.
	"""
	return render_template(
        "index.html",
        dataset=dataset,
    )

My index.html has this line to receive the data (the parameter dataset should match between the 2 files). This actually took 5ever to figure out…

<script>var data = {{ dataset|tojson }};</script>

The following line in the HTML script uses the data variable to construct the visualization.

Repositioning Flowers with D3

While I didn’t have to change much of the core logic in my visualization script, I had to find a new way to generate flower positions dynamically. D3’s simulation has forceCollide and forceCenter (or forceX and forceY) that would help center the flowers while avoiding collision.

My result below accomplished the spaced-out flowers that I wanted, but introduced a new problem: the x-axis was no longer ordered. I created a flower stems that showed what each section’s order was, and this randomness from the simulation made it look tangled and messy.

I changed the simulation so that it would position flowers along the x-axis before centering them. I had some trouble with their starting positions because D3 would ignore the x and y positions I set for my group elements before running the simulation. This is why all the stems unintentionally pointed towards the (0, 0) coordinate below. I also replaced the flowers with black circles because the flowers' gaussian blur filters made everything lag.

There were 2 ways to set the starting positions along the x-axis:

1. Set the elements' position attributes using d.fx instead of d.x. D3’s source code has a method called initalizeNodes that would check if the node positions should be fixed. I would set the fixed position and then make it null after the simulation’s alpha (“entropy level”) is under some threshold.
2. Create forces that would set each flower’s gravity toward’s their respective x-position, remove that force, and then reset the simulation with an altered center force. This is like running 2 simulations one after the other.

I went with option #2 and coded everything within the d3.simulation.on("tick") block. There were 3 possible actions coded as an if statement:

1. If the simulation was still active, continue running.
2. Else, if we are on Simulation 1 of 2, then restart the simulation and change the forces to reflect Simulation 2 of 2.
3. Else, stop the simulation and draw the remaining visual.

My code is generalized below.

var iteration = 1; // We are on Simulation 1 of 2
var simulation = d3.forceSimulation(data)
    .force("x", d3.forceX().x(d => d.x)) // Gravitate toward their x-positions
    .force("collide", d3.forceCollide().radius(d => d.radius)) // Avoid overlapping
    .on("tick", () => {

        if (simulation.alpha() > 0.1) {
            flowers
                .attr("transform", d => `translate(${d.x},${d.y})`); // continue running simulation

        } else if (iteration == 1) {
            simulation.alpha(.5).restart(); // restart simulation with new forces
            simulation
                .force("center", d3.forceCenter().x(width/2).y(height/4)) // Modify this force
                .force("x", d3.forceX().x(width/2)); // Create this new force
            iteration = 2; // We are moving to Simulation 2 of 2

        } else {
            simulation.stop(); // stop simulation
            doRemainingStuff(); // run the remaining steps AFTER simulation
        };
    });

Now the flowers and stems are positioned where they should be thanks to timing forces correctly!

In case you were wondering why I didn’t choose option #1, I actually didn’t know of it at the time! In retrospect, I think it would have shaved a few ticks off the clock if I had fixed their starting positions. :P

Deploying on Heroku

I deployed on Heroku because the service has a free tier and several examples of deploying Flask applications. The requirements are pretty simple once you have done it at least once, and it does most of the infrastructure work for you. Following this guide, I created a requirements.txt, Procfile (Heroku-specific), and runtime.txt (also Heroku-specific).

One difference I had with the guide was that I added my main script in an app subdirectory. I specified the script’s location with a --pythonpath app tag in my Procfile.

# Conventional command in Procfile
web: gunicorn run:app 

# My command in Procfile because I had an `app` subdirectory
web: gunicorn --pythonpath app run:app

My Web Application

Try your favorite Spotify song here! I like showcasing Hamilton because the bouquet is very rich like the song. :P