Blog: [Blog Home] [Archives] [Search] [Contact]

Posts Tagged ‘Processing’

Pierre-Auguste Renoir The Umbrellas Algorithmic Interpretation Video

Saturday, December 2nd, 2017

Pierre-Auguste Renoir The Umbrellas Algorithmic Interpretation Video
Pierre-Auguste Renoir The Umbrellas Algorithmic Interpretation Video

I just created a video for the first time in quite a while for my YouTube account. It came about when I was experimenting with algorithmic representations of a source image using the Processing programming language. The image I happened to be working with was “The Umbrellas”, a painting by Pierre-Auguste Renoir.

As I worked with variations on the algorithm it struck me that an animation could be interesting. The basics of the algorithm were to first create a grid of equally spaced points and assign each a x,y location. The key variable at this point was the amount of spacing there would be between each x,y grid location. For this video I set the grid spacing to be 10 pixels.

Next was to get the color value from Renoir’s “The Umbrellas” image for each of the x,y locations in the grid. To draw I had decided to use the line() function. The variables for each line would be color, starting angle, and length. For the line’s color, I simply used the color value taken from the image for that location. For the starting angle of rotation, I opted to use the hue taken from the corresponding pixel. To get the hue required converting the RGB color value into its HSB (hue, saturation, brightness) equivalent. The length of the each line being drawn was determined by the brightness of the pixel at that location. This resulted in another variable: establishing what the maximum line length would be. So brightness values of 0 to 255 had to be translated into a range of lengths from some minimum value to some maximum value. The standard way of doing this in Processing is to use the map() function. I never use map() because it is an inefficient way to translate numbers from one scale into another scale. For the video, I simply divided the pixel brightness by 12 – meaning the longest a line could be was 21 pixels.

To add some variability, I added a variable for the Z axis and used a Perlin noise field to control each location’s Z coordinate. The result is that the distance of each line from the camera varies somewhat, which enhances the perception of depth in the image.

To animate the image required changing one or more of the variables associated with each grid point over time. Keeping things simple, I added a global variable that would equally increment each line’s rotation angle between frames. This created a uniform rotation for all the lines.

I then added a time variable to alter the Perlin noise field values over time and updated each line’s Z coordinate between frames. The main issue here was with respect to how much I wanted the range of Z values to vary. For comparison, below is an illustration of what you would see if a substantially greater range of Z coordinate values was allowed.

Pierre Auguste Renoir The Umbrellas Algorithmic Interpretation Video Variation
Pierre Auguste Renoir The Umbrellas Algorithmic Interpretation Video Variation

To create the output, I used the Processing saveFrame() feature to write each frame of the movie to a tiff file. Separately I had used Audacity to create a narration soundtrack for the video. Once I knew how long my audio track was, I simply dropped a variable into my Processing program which indicated the frameCount at which to stop generating image frames.

While I have previously used tools like FFMPEG to create videos, this time I decided to use Processing’s Movie Maker tool. Confirming that my tiff and mp3 files were fine, I started up Movie Maker. I specified the input sources for my files, went with the default compression of animation and clicked the “Create Movie” button. I then monitored the dialog window as the program progressed through the 3000 images used to create the video. The program ended without error.

But when I went to view the video using VLC, all I got was a black screen and horribly garbled and spotty audio. I had no idea what had gone wrong. Rather than resorting to one of my other tools, I opted to give Movie Maker another try. The only difference was that this time I selected the JPEG option for compression. The dialog proceeded as before and again ended without error. This time the video and audio were fine, except for a narrow strip of color along the video’s edge. For purposes of this video, that is something I can live with.

Unfortunately the image quality of the video has suffered due to YouTube’s overly enthusiastic image compression. Whereas my original video upload was a 3.1 gigabyte file, YouTube compressed it down to a mere 29 megabytes (you can’t throw away that much information without losing quality). While I do understand the need to economize on bandwidth, such economies can be achieved in part by viewing the video at one of the lower resolution settings.

You can view the video on YouTube at: https://youtu.be/CNE0j1LXIJ0. Give the video a watch, let me know what you think, and share it if you like it.

 

Bookmark it:  Stumble It  Bookmark this on Delicious  Digg This  Technorati  Reddit Tweet It


Creative Coding Software Tools: Processing, openFrameworks, Cinder

Thursday, April 14th, 2016

Creative Coding Software Tools:
Creative Coding Software Tools: Processing, openFrameworks, Cinder

In my previous blog post, Fresh Brewed Coffee Digital Art, I made mention of the fact that I create my digital art using software of my own design and that for those digital artists interested in pursuing this aspect of digital art creation, there were some alternative tools available. In that post I mentioned Processing, openFrameworks, and Cinder. I would like to take this opportunity to say a little more about each of these three options.

Processing

Starting with Processing, this is a framework and programming language that is built on top of Java, an object-oriented programming language. Like Java, Processing is free and available on a variety of platforms. Personally I use Processing on both Windows 7 and Ubuntu Linux. Because the Processing programming language was created for artists and musicians with little or no programming background, beginners can quickly be up, running, and creating with this wonderfully flexible software tool. The flexibility of Processing as an environment for creative coding is expanded by the abundance of third party libraries that have been made available. It is also the most flexible tool in terms of the variety of platforms it works with. I have taken advantage of the ability to write Processing sketches for the web using the Javascript version of Processing (Processing.js) as well as for creating Android apps and for interacting with the Arduino (see The Arduino Starter Kit – Official Kit from Arduino with 170-page Arduino Projects Book). For those new to programming and creative coding, Processing is my number one recommendation.

Processing Resources

The main Processing web sites are:

Following are three books on Processing that I recommend and own. There are a number of other books on Processing that are also quite good. Please be aware that Processing is now on version 3 and version 2 is still widely used but do avoid any book that was written for version 1 of Processing.

openFrameworks

Like Processing, openFrameworks is also free and available on multiple platforms. In fact I even had the opportunity to write some openFrameworks programs on a Raspberry Pi (see CanaKit Raspberry Pi 3 Ultimate Starter Kit – 32 GB Edition) that was running the Raspbian operating system. The primary difference between Processing and openFrameworks is that whereas Processing is a framework that sits on top of the Java programming language, openFrameworks sits on top of the C++ programming language. Personally I find openFrameworks to be somewhat more challenging than Processing, particularly with respect to the use of off-frame buffers in conjunction with OpenGL. And by challenging, I am speaking in terms of the number of lines of code I must write in order to achieve some objective.

openFrameworks Resources

The main web sites for openFrameworks are:

There are not nearly as many books about openFrameworks as there are about Processing but the two that are most worthwhile are:

If you are searching on Amazon for books about Processing and/or openFrameworks, you may come across the book Programming Interactivity: A Designer’s Guide to Processing, Arduino, and openFrameworks by Joshua Noble. My advise is do not buy this book. It is quite out of date and the source code for the examples never was made available.

Cinder

Cinder is a third creative coding platform and, like openFrameworks, relies on the C++ programming language. I have no personal experience with Cinder but I will say that when I was investigating openFrameworks vs Cinder as a creative coding toolset for the C++ environment, openFrameworks won out.

Cinder Resources

The main Cinder web sites are:

There are even fewer books about Cinder than there are about openFrameworks. Two books you will find on Amazon are:

I hope you’ve found this information useful. I also hope that, even if you are not a digital artist or musician or programmer, you check one or more of these creative coding toolsets because you never know – you just might have a knack for creative coding.

Bookmark it:  Stumble It  Bookmark this on Delicious  Digg This  Technorati  Reddit Tweet It


Makers Faire Milwaukee, Charon Wallpaper, and Art Workflow

Saturday, October 10th, 2015

Inside Makers Faire Milwaukee
Inside Makers Faire Milwaukee

A couple weeks ago I attended Makers Faire Milwaukee. I had meant to do a write up of my experience but I’ve had a case of writer’s block in figuring out just what I wanted to say. I was at Makers Faire for two reasons. First, I was conducting a workshop on Creating Digital Spirographs and Harmonographs with Processing – a favorite of mine as it combines art with math with programming.

Second, wearing my NASA Jet Propulsion Laboratory Solar System Ambassador hat, I delivered a presentation on the New Horizons mission to Pluto. With respect to my Pluto talk, a number of the illustrations I use are either my own renders or my own interpretations of New Horizons imagery. For example, here is a free wallpaper version of my enhanced and colorized image of Charon that I’ve previously shared on social media.

Lastly I’ve been adding new fields to my art inventory spreadsheet that I hope to be able to use to help automate the production of gallery pages here on my web site. While I was at it, I decided to do a basic write-up of the data keeping portion of my art work flow. If this is of interest to you, then see My Art Workflow – The Management Bits.

With respect to my art, consider checking out my portfolios on the following art print on demand sites.

Bookmark it:  Stumble It  Bookmark this on Delicious  Digg This  Technorati  Reddit Tweet It


Video: The Liquified Paintings of Claude Monet

Monday, July 7th, 2014

Liquified Paintings of Claude Monet
The Liquified Paintings of Claude Monet Video

Since setting up an account on YouTube towards the end of last year, I confess to not having been active on that platform. I created the account for the purpose of publishing several video portfolios to promote my art. The plan was to create a video for each area of artistic creation I am working in. I created exactly one portfolio video and that was for my portrait art. That was my first attempt at making a video and you can see it here: Portrait Art Video. If you’re interested in the story of how I went about making that video, read Portrait Art Video Project.

The experience of creating that video got me interested in creating some original animations of my own. Since that time I’ve only posted two videos exploring animation. One I dubbed the Swimming Eye Art Video. The other was a crude quickie experiment in animating an image – Sailing A Stormy Sea Video.

For this new video I wanted to create something that would feature the art of the great impressionist painter Claude Monet. I have recently been experimenting with vector fields and their utility as an algorithmic means of creating flowing brush strokes. It occurred to me that I could use this technique to create a series of liquified paintings that would evolve. And that’s how The Liquified Paintings of Claude Monet video was born. And here it is.

The video captures the evolution of six separate paintings and the transition from one to the next. For me it is the transition between paintings that is visually the most interesting. One thing you may have noticed is the very slow evolution of the first painting. It is no coincidence that this first painting is the darkest of the six paintings. You see I tied the speed of evolution to the overall brightness of the image.

Given that these paintings have been "liquified", I deliberately chose artworks by Monet that featured water, be it a pond, a stream, a river, or the ocean. Following are image stills from the video and the name of the Monet painting that was used as the color source at that point during the video.

Claude Monet Impression, Soleil Levant
Claude Monet – Impression, Soleil Levant

Claude Monet The Argenteuil Bridge
Claude Monet – The Argenteuil Bridge

Claude Monet Morning By The Sea
Claude Monet – Morning By The Sea

Claude Monet Autumn On The Seine At Argenteuil
Claude Monet – Autumn On The Seine At Argenteuil

Claude Monet Poplars At The Epte
Claude Monet – Poplars At The Epte

Claude Monet Water Lilies
Claude Monet – Water Lilies

The most time consuming aspect of this project was writing the program that produced the video stills. In all I used 3272 image stills (not counting the title and trailer images) to create this video.

Graphics Software Used

I created this video using several different software packages. The liquified/animated images used to construct the video were created with a program I wrote using the Processing creative coding platform which is a framework built on Java. As a programming language, Processing is easily the best language for non-programmers interested in creative coding projects. To stitch the individual images together into a video, I used the DOS command line utility FFMPEG. To create my title and trailer images I used Adobe Photoshop CS4. Note that my workflow would have consisted entirely of "free" software if I had used GIMP (GNU Image Manipulation Program) to create these two images. For the soundtrack file, I used Audacity to edit the mp3 sound file.The soundtrack music is Laideronnette Imperatrice Des Pagodes by Maurice Ravel. Finally to assemble everything I used Microsoft’s Windows Live Movie Maker which came bundled with Windows 7.

In Conclusion

If you would like to know more about Claude Monet, you may want to read this biography of Claude Monet. If you are of a technical bent, there is this Wikipedia entry for vector fields which served as the painting foundation upon which my Processing program was built. And while I don’t often add new videos, you may want to follow me on YouTube.

I’ll close with a couple of noteworthy quotes.

When you go out to paint, try to forget what objects you have before you – a tree, house, a field….Merely think, here is a little square of blue, here an oblong of pink, here a streak of yellow, and paint it just as it looks to you, the exact color and shape, until it gives your own naive impression of the scene before you. – Claude Monet

A preliminary drawing for a wallpaper pattern is more highly finished than this seascape. – French art critic Louis Leroy in 1874 commenting on Monet’s Impression, Sunrise

Bookmark it:  Stumble It  Bookmark this on Delicious  Digg This  Technorati  Reddit Tweet It


Image Processing and Telling RGB/HSB Color Lies

Thursday, May 8th, 2014

Squashed Blue Man Statue
Squashed version of Blue Man Statue digital painting for testing

As a practitioner of digital art and image processing, and with a background in both math and computer programming, I regularly create my own graphics programs using the Processing programming language. Pictured at the top of this post is a squashed version of a digital painting I did using Adobe Photoshop and some custom brushes I had created. Pretty straight forward stuff.

Recently I’ve been exploring the world of generative art creation by writing my own generative art programs. For some of these programs, rather than starting with a blank canvas I provide an initial image from which to work. The image may be a photograph or a work of digital art. For example in one instance I took a selfie (a self-portrait photograph), created a painted version of that photograph and fed that into one of my generative art programs. (Note: you can see the resulting artwork Generative Selfie at Artflakes.com.)

Unfortunately with large size images complex generative programs can take quite a while to run. Consequently I use whatever tricks I know to speed up program execution. One common ‘trick’ is to avoid using Processing’s canned color routines and to use bit-shifting instead. Bit-shifting allows for very speedy access to an image’s color information which is encoded in an RGB (red,blue,green) format. This means that color is defined by the three values of red, green, and blue. Bit-shifting works because the four individual values for red, green, blue, and alpha (transparency), are all stored in a single 32-bit integer field.

The other night I thought of a cool modification that I could make to one particular generative art program I’ve been working on. However that change would require that I work in HSB (aka HSV) mode. With HSB/HSV, a color is defined by the three values of hue, saturation, and brightness (sometimes referred to as value). Working programmatically in RGB has several drawbacks when compared to the competing HSB color model. HSB provides much more flexibility when it comes to creatively manipulating color.

There is just one problem with the HSB approach. The color information in images is stored in RGB format. The bit-shifting method that works so nicely is not an option for working with HSB. There are standard routines that allow you to extract HSB information from the RGB color format but you pay a penalty in terms of the amount of processing time it takes to do that. And if you are working with an image that has tens of millions of pixels and you are performing a lot of color sampling, let’s just say that your computer is going to be tied up for a while. My back of the envelope calculation leads me to believe that working with HSB would result in an additional 50 million-plus program statement executions in my code and an unknown number of additional statement executions in the underlying Processing and Java code.

By nature I’m an impatient person so for me all this additional program overhead was unacceptable. And then it dawned on me – I could LIE! You see computers are stupid and will believe whatever you tell them. As supporting evidence I offer up the views of science fiction author Arthur C. Clarke:

…the fact is that all present computers are mechanical morons. They can not really think. They can only do things to which they are programmed.

The LIE that came to me was to write a Processing program that would take all the RGB color information from an image file and replace it with HSB information. I could then use that modified version of the image file as input to my HSB generative art program and it would run just as fast as the original RGB version because I would be able to use those very efficient bit-shifting operations. While I was at it I also wrote a utility that converted the file from HSB back to RGB. This allowed me to visually compare the original image with an image after it had undergone the RGB to HSB and back to RGB conversions.

Of course the downside of stuffing HSB data into the RGB field is that every other program on my or anyone else’s computer is going to read that image file and expect that the color information is in RGB format. Take a look at Image 2 below. It’s a copy of the file shown above except I’ve put HSB information into the RGB field. Kind of cool.

Appearance to RGB-reading software
Image 2. How the image looks to RGB-reading software when the file actually contains HSB information.

Taking this whole lying idea a step further, what if I lie to my color converting utility? What if I do the same RGB-to-HSB conversion multiple times while throwing in a few HSB-to-RGB conversions as well? What you can wind up with is one confused picture. Image 3 is an example of the kind of image you can get. In fact you could argue that Image 3 is more artistic than the original painting.

multiple random RGB-to-HSB and HSB-to-RGB conversions
Image 3. Running multiple, random RGB-to-HSB and HSB-to-RGB conversions.

Pablo Picasso once observed that art is a lie that makes us realize truth. That may be but in this case a lie was simply the most expedient way to achieve an artistic objective. Having spent all this time coming up with a nice RGB-to-HSB color conversion utility, it’s now time to get to work on the HSB version of that generative art program.

References

For those of you who would like to know more about RGB, HSB, and Processing, you can check out the following references.

Bookmark it:  Stumble It  Bookmark this on Delicious  Digg This  Technorati  Reddit Tweet It


Generative Art and the Mona Lisa Meme

Friday, March 28th, 2014

Generative Art and the Mona Lisa
Generative Art from the Mona Lisa
left to right: Generation 3922, 5826, and 8187

I want to share with you the results of a recent experiment of mine using a creative process known as generative art. Personally I find that the most interesting aspect of generative art is in being surprised by the artwork that a system produces. Generative systems can produce artistic outcomes not anticipated by the programmer/artist and the image above is one such example. On the left is an image of the Mona Lisa as it appears after 3,922 generations of the generative art program I wrote. On the far right is the same image after 8,187 generations.

What is Generative Art?

For the purposes of my discussion here I’ll rely on an excerpt from the Wikipedia definition of generative art:

Generative art refers to art that in whole or in part has been created with the use of an autonomous system. An autonomous system in this context is generally one that is non-human and can independently determine features of an artwork that would otherwise require decisions made directly by the artist… Generative Art is often used to refer to computer generated artwork that is algorithmically determined.

Source: Wikipedia definition of Generative Art

Why are you picking on the Mona Lisa?

When testing out various programs that rely on the use of a source image for input it is quite useful to have a single standard image to use. That makes it much easier to compare the workings of different programs. An analogy is that of Playboy centerfold and Swedish model Lena Söderberg. Lena was the centerfold for the November 1972 issue of Playboy magazine. Her centerfold photograph was first used as a test image for image processing experiments in the summer of 1973 at the USC Signal and Image Processing Institute (SIPI). Subsequently this photograph became a standard source image for the testing of image processing algorithms. In explaining the decision for the use of this image, David C. Munson, editor-in-chief of IEEE Transactions on Image Processing, had this to say:

"First, the image contains a nice mixture of detail, flat regions, shading, and texture that do a good job of testing various image processing algorithms. It is a good test image! Second, the Lena image is a picture of an attractive woman. It is not surprising that the (mostly male) image processing research community gravitated toward an image that they found attractive."

My test image of choice is Leonardo da Vinci’s painting Mona Lisa (La Gioconda in Italian). Because this painting is so well known and accessible, it makes it easier for people to "see" the results of an manipulation, distortion, or derivation of the original.

My Oscillating Generators

The generative art program that I wrote, which produced the illustrations at the head of this post, relies on a system of generators. You can think of each generator as simply being an independently functioning paintbrush.

In this particular run, I used 9200 generators (paintbrushes). Each generator (brush) has several characteristics: size, location, color, opacity, movement. In creating this system my idea was that each paintbrush would hover in the vicinity of its original location without straying too far. However, I did not provide a rule to enforce this behavior. Rather I left each brush free to go its own way.

To govern direction and speed I used a Perlin noise function that on the face of it was balanced. By balanced I mean that the system should have had no preferential direction. I was very much surprised at the results (shown above) from one of the several rule sets I had created.

For simplicity, each generator is unaware of the other generators in the system. For the next generation of this system, I plan on creating interacting generators. In such a system, when two generators encounter one another, they will react and/or interact. For example each could share with the other some or all of its characteristics. Each of these characteristics can be thought of as genetic material that can be shared.

So that you can better see the detailed progression of the system, I’m providing a large (1600 x 1600) image that shows the same subsection of the artwork as it progresses through generations. The leftmost section is from generation 3922, the middle section is from generation 5826, and the rightmost is from generation 8187.

Open image in new window – Generative Art Mona Lisa Triptych

For other examples of how the image of Mona Lisa has been used, check out the Processing artwork tagged with Mona Lisa at OpenProcessing.org

Bookmark it:  Stumble It  Bookmark this on Delicious  Digg This  Technorati  Reddit Tweet It