Tool that allows people to pick up any object in their environment and use it to scroll through PDFs
Self-directed experiment
Processing
During my internship at Canonic, Simranjot once floated around an idea of how convenient it would be to be able to sit back and scroll through articles without leaning into a laptop.
I found his thought intriguing and decided to make a prototype for him on Processing. You can access the Java applet here.
A short explainer video of sorts. Unmute for some nice beats :)
The program uses Daniel Shiffman's blob detection algorithm. The program intiially asks the user to click on the colour that needs to be tracked. The x & y locations as well as the RGB values of the selected pixel are then stored.
void mousePressed() {
// Save color where the mouse is clicked in trackColor variable
int loc = mouseX + mouseY*video.width;
trackColor = video.pixels[loc];
}
The program then walks through every pixel of the webcam video to match all the pixels that result in a similar RGB value as stored in the trackColor variable. A blob is then added to the cluster of identified objects, which will move along with the real world object's x & y position in the webcam video.
// Begin loop to walk through every pixel
for (int x = 0; x < video.width; x++ ) {
for (int y = 0; y < video.height; y++ ) {
int loc = x + y * video.width;
// What is current color
color currentColor = video.pixels[loc];
float r1 = red(currentColor);
float g1 = green(currentColor);
float b1 = blue(currentColor);
float r2 = red(trackColor);
float g2 = green(trackColor);
float b2 = blue(trackColor);
float d = distSq(r1, g1, b1, r2, g2, b2);
if (d < threshold*threshold) {
boolean found = false;
for (Blob b : blobs) {
if (b.isNear(x, y)) {
b.add(x, y);
found = true;
break;
}
}
if (!found) {
Blob b = new Blob(x, y);
blobs.add(b);
}
}
}
}
for (Blob b : blobs) {
if (b.size() > 2) {
b.show();
}
}