mehan's itp blog

A blog documenting my work at NYU ITP. Visit my main site at mehanjayasuriya.com for more information.
hello! theme by cissysaurus

The State of Mobile Broadband in New York State

image

For my final project for Data Without Borders and Understanding Networks, I decided to analyze broadband speed data and per capita personal income data and see if I could find a relationship between the two. Initially, I had planned to analyze data for the entire country but as it turns out, that’s a lot of data. The mobile broadband data set for the nation comes out to about 11GB, which is impractically large to use with R or most mapping solutions. I decided to limit my scope to mobile broadband networks in the state of New York, just to make things easier.

For the broadband data, I went to the FCC, which provides broadband speed data broken out by state and type (e.g. fixed vs. wireless). I had hoped to use actual observed speeds but the columns that were supposed to contain that data were blank, so I had to settle for maximum advertised speeds (which, as you probably know, tend to be quite optimistic). For the income data, I used the New York State census results from 2010. Despite some inconsistencies in format, both data sets contained a 5-digit FIPS number, which is used to identify census tracts. Using R, I merged the two sets using the FIPS number.

Before I went any further, I decided to do a bit of analysis. I went ahead and created a plot of income vs. wireless download speed:

Personal income vs. Wireless download speed

As you can see, the data separated pretty cleanly into “buckets”. That’s because advertised broadband speeds generally tend to be whole numbers—most carriers will advertise 8Mbps down as opposed to, say, 8.43Mbps. I ran a jitter on this graph but that didn’t make it any easier to read. So I went ahead and plotted a linear regression line on the graph:

image

As you can see, the linear regression line shows a positive correlation between income and broadband speed. But is the relationship legit? Here’s the summary stats that R gave me on the regression line:

Call:

lm(formula = newmerged$wirelessdl ~ newmerged$income)

Residuals:

    Min      1Q  Median      3Q     Max 

-6.0631 -1.7524  0.1637  1.8632  4.4579 

Coefficients:

                  Estimate Std. Error t value Pr(>|t|)    

(Intercept)      4.019e+00  2.389e-02   168.2   <2e-16 ***

newmerged$income 8.262e-05  6.350e-07   130.1   <2e-16 ***

—-

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 1.992 on 217558 degrees of freedom

Multiple R-squared: 0.07219, Adjusted R-squared: 0.07219 

F-statistic: 1.693e+04 on 1 and 217558 DF,  p-value: < 2.2e-16 

Based on my extremely rudimentary understanding of statistics, that really low P-value (2.2e-16) suggests that this correlation is probably the real deal.

After establishing the connection between income and speed, I decided to visualize the data using CartoDB. I fed my data into CartoDB and had CartoDB resolve geocodes (latitude and longitude) from the city names in the income data. I’m guessing that they simply make a call to the Google Maps API for each point, in order to get the geodata.

image

Finally, I plotted all of the points on the map and used CartoCSS to style the map, which took a lot longer than I had anticipated. Not only are there two variables for each row in the table (income and download speed) but for many locations on the map, there are multiple data points, as most areas are served by multiple wireless providers. I tried to tweak the styles in such a way that would make the relationship between the variables clear, which was pretty challenging. Here is the end result.

After finishing the map, I realized that it might have made more sense to just display the highest download speed for each point rather than all of the various speeds, just to make the map a bit more readable. I decided to go back to R and limit the data to just the fastest download speed for each census tract. In order to do this, I used a command in the “plyr” library called “ddply”: 

fcc4 = ddply(fcc3,~FIPS,function(x){x[which.max(x$fcc2.maxaddown),]})

Currently, R is chugging along, trying to apply that function to my 340,144 rows of data. After that’s done, I’ll have to feed the new table to CartoDB and have it re-geocode all of the data, which could also take a while. I’ll update this post if I’m able to finish the second map in time—it would be nice to see them side-by-side to see which one is more readable.

This week in our Understanding Networks class, Tom Igoe tasked us with designing a device that can serve as a client for a simple online game he had set up (essentially a multiplayer version of “Pong” running on a webserver). I decided to use an Arduino Uno, an ethernet shield and an NES controller to build a rudimentary game console. The NES controller, as it turns out, pairs pretty well with an Arduino, since it requires only three digital pins to transmit data. I used the NESpad library to read the controller signals (if you want to use NESpad with the Arduino 1.0+ IDE, I’d recommend using this patch) and Tom’s client code for opening the socket connection and sending the controller data to the webserver. Now that I’ve got this up and running, I will likely consider using the NES controller as an Arduino interface for future projects.

mehan:

I drew some pictures of foxes for my Animal Studies class last semester.

HTML 5 FISHTANK

My week 4 homework for HTML 5. No fish were harmed during the making of this novelty web application.

MEMEMAKER

This is my week 4 homework assignment for Dynamic Web. Built using NodeJS, Express and Bootstrap. Based off of John Schimmel’s Valentine Card Maker.

Animals week 3 assignment: the octopus in visual culture

The octopus in visual culture (text) (PDF)

The octopus in visual culture (slides) (PDF)

Animals Week 2 Homework
For my second homework assignment, I wanted to see if I could do a lot with a little. I&#8217;ve always admired the way that great character designers are able to create expressive characters without leaning on any of the visual cues we use to read human emotions (think: the lamps in Pixar&#8217;s Luxo Jr.), so I wanted to see if I could do something similar with my octopus character design. Of course, in order to communicate emotion using a character on a static page, I needed to add something, so I chose to anthropomorphize my octopus enough to add a set of eyebrows. I am pretty sure that octopuses don&#8217;t actually have eyebrows.

This octopus lives on the ocean floor. While it leads a fairly solitary life, it interacts, from time to time, with other bottom-dwellers like eels, crabs, lobsters and other octopuses. While it is adept at catching smaller prey, it is equally adept at hiding itself from predators (like sharks and humans), using both camouflage and its ability to squeeze into tight spaces.

Animals Week 2 Homework

For my second homework assignment, I wanted to see if I could do a lot with a little. I’ve always admired the way that great character designers are able to create expressive characters without leaning on any of the visual cues we use to read human emotions (think: the lamps in Pixar’s Luxo Jr.), so I wanted to see if I could do something similar with my octopus character design. Of course, in order to communicate emotion using a character on a static page, I needed to add something, so I chose to anthropomorphize my octopus enough to add a set of eyebrows. I am pretty sure that octopuses don’t actually have eyebrows.

This octopus lives on the ocean floor. While it leads a fairly solitary life, it interacts, from time to time, with other bottom-dwellers like eels, crabs, lobsters and other octopuses. While it is adept at catching smaller prey, it is equally adept at hiding itself from predators (like sharks and humans), using both camouflage and its ability to squeeze into tight spaces.

Animals Week 1 Homework

The blue-ringed octopus is a manually dexterous creature. It is able to easily manipulate objects with its eight arms and suction grip. Docile unless provoked, the blue-ringed octopus is a threat to any animal that would try to attack it: it produces enough neurotoxin to kill 26 adult humans. In captivity, octopuses are known to be quietly crafty&#8212;they will sometimes sneak out of their enclosure at night, devour animals in other tanks and return to their own tank without leaving a trace.  

Animals Week 1 Homework

The blue-ringed octopus is a manually dexterous creature. It is able to easily manipulate objects with its eight arms and suction grip. Docile unless provoked, the blue-ringed octopus is a threat to any animal that would try to attack it: it produces enough neurotoxin to kill 26 adult humans. In captivity, octopuses are known to be quietly crafty—they will sometimes sneak out of their enclosure at night, devour animals in other tanks and return to their own tank without leaving a trace.  

Dynamic Web Week 3 Homework Assignment

Dynamic Web Week 2 Homework Assignment

dynamic web project idea: instagram —> tumblr

For the last few days, I’ve struggled to come up with a compelling idea for a project for my Dynamic Web class. Ultimately, I decided that instead of worrying about what might be useful/interesting/zeitgeisty, I should just build a web application that I myself would use. So here’s the idea that I settled on: an application that automatically sends posts that appear in your Instagram stream to your Tumblr dashboard. While Instagram has an active community, at present, there’s no way to browse your Instagram stream other than via the official iPhone app and a handful of unofficial (and oftentimes buggy) web clients. Tumblr, meanwhile, has built an easy-to-use dashboard that’s ideal for browsing content in reverse-chronological order and my sense is that their userbase has a good deal of overlap with Instagram’s. My idea is to create a service that would ask the user to authenticate with both his/her Instagram and Tumblr credentials and would in exchange, automatically send content (i.e. photos and metadata) from the user’s Instagram stream to Tumblr. I’ve poked around a bit in the documentation for the Instagram and Tumblr APIs and while it’s not yet clear if it will be possible to build this sort of thing, hopefully, I will be able to figure out a way to make it happen over the course of the next few months.

Photo of me demoing The 4D Pop-Up Book of Halloween at the 2011 ITP Winter show. Thanks to Spike McCue for the photo!

Photo of me demoing The 4D Pop-Up Book of Halloween at the 2011 ITP Winter show. Thanks to Spike McCue for the photo!

Above you’ll find a video demo for The 4D Pop-Up Book of Halloween, an augmented reality children’s book that I built as part of my research at NYU ITP. It’s a handmade pop-up book that’s embedded with a different QR code on each page. When the reader scans the codes using a webcam-equipped device and a companion Mac/Windows/Android application, it produces an animation onscreen that places the reader inside the story. My goal was to build a book that could bridge the gap between one-of-a-kind artifacts and purely digital experiences. This project was featured in the 2011 ITP Winter Show.