→ a breakthrough
Hello, reader! Thank you greatly for perusing this publication. This is a piece I wrote last semester at a moment I felt I had my first real conceptual breakthrough on my Thesis project. My project was always centered around Machine Intelligence and creativity, but the project I’ve ended up with is far, far different than what is described or speculated on here; (for more info on that, search my name or my project, “Object Americana,” in our online Thesis gallery!). This piece stands alone as a sort of snapshot of the feeling/thinking/doing/ego/uncertainty/tumult of MFADT Thesis.
→ a breakthrough
Last night I believe I had a breakthrough. I had just spent the night winning the majority of my team’s points for the evening at PinballNYC’s bar league (week 4). I am, as they say, finally coming into my own as an A-league player. Very satisfying. During the evening, the bartender was playing the TV show ‘Antique Roadshow,’ which features random americans bringing in their various antiques to be appraised by experts. A man brought a humongous wooden vase, apparently created by midcentury american wood-turner Ed Moulthrop. It was especially notable for being both genuine and remarkably huge, requiring a gigantic lathe and likely custom-built tools to carve. It had sadly suffered some humidity damage in the owner’s garage, and as such the price was appraised at $17-$20,000 instead of something like 40 grand. Quite striking. It struck me particularly because I know something about making things (many people commented on how impressive the piece was, and they weren’t wrong), yet this piece wasn’t quite my taste, being wooden.
Earlier in the day, I had my first one-on-one meeting with my thesis advisor, Anežka Šebek, PhD. Our discussion went quick and wide-ranging, she emphasized some more sources for me to look at, but the biggest takeaway I had was that I need to find a niche within the rather broad field I’m looking at, and crawl in. What is the thing I could do to contribute to the advancement of the field of creative AI? Why? How?
As I was laying in bed turning the events of the day over in my head, I was thinking again about Ed Moulthrop and how highly people value the craft of his work. This is a relevant discussion to me for many reasons, not least of which is my colleague, Bolor Amgalan, who is deeply focused on the concept of craft and what it means for the contemporary era in the context of virtual reality. As someone with a long and substantive background in physically making things, I am also quite interested in the dichotomy between digital and manual manufacturing, and the extent to which manual craft-making is at once highly prized and somewhat denigrated compared to its digital counterparts. I personally tend to find myself on the ‘side’ of digital manufacturing (so much so that I coined the term digifacturing; ‘manufacturing’ literally means ‘to make by hand’).
It struck me that one particular joy a person like Ed Moulthrop must’ve had in their art is that no two pieces they made would ever be the same. Even if you used the same plan or jig for a wooden bowl, the nature of the craftwork entails that the final product is truly unique, not least because there is tremendous emphasis on the pattern contained in the wood itself. It is not a revelation to say that one reason craft-making is so highly valued is because it results in unique pieces, rather than factory-made replications of the same design. I was then given pause by the thought that 3D printing is a fascinating sort of in-between of these two methods; if one is using a good, well-tuned machine with the same filament repetitively, one should get repetitive, nearly-identical results. Yet the vagaries of the machines and the difficulties in consistency can often mean that 3D printed objects do indeed have a bit of uniqueness to them, whether it’s a small flaw or an odd surface ‘artifact’ that arises from a combination of the printing script file and the filament quality. Further, there are now 3D printing filaments (the raw plastic stock, produced in kg-size spools of hundreds of meters) which can contain multiple colors, thus making each piece printed with them inherently unique regardless of machine quality.
However, the ideal in 3D printing remains that a faithful machine will faithfully reproduce a given gcode (the raw scripting file that tells the machine what to do) file with 100% consistency, seemingly abrogating the notion of uniqueness-of-object that ‘craft’ rests so heavily upon.
It was then that I had a breakthrough in thought. What if I managed to use a generative text-based Machine Intelligence system to attempt to reproduce the gcode files themselves?
This is a highly provocative idea for a variety of reasons. Gcode is an extremely simple scripting code; it can hardly even be called a language in the traditional sense of coding. It merely comprises a series of commands, each with their own number-code. The vast majority of the commands are simple position coordinates; you are telling the machine tool-head where to go sequentially. I emphasize this because the bulk of the work being done in Machine Intelligence (MI) development tends to be in NLP, Natural Language Processing. This is for a multitude of reasons but the central one is probably the fact that we as language-using, sentient creatures seem to have language as our basis of sentience. That is to say, we structure our thoughts as a sort of language-like set of meaning-relations. This structure has rules that develop meaning; the relationship between a noun and a verb is fundamental to our understanding of events and time/consequences in the world.
Programming languages likewise have grammar and syntax rules and they tend to be far more strict than ‘natural’ language. I assume this is a crucial reason why the notion of MI-generated programs hasn’t really caught on yet [CITATION NEEDED]. However, gcode is again dreadfully simple. It also has a fundamentally physical aspect to it, something that most MI-generated art famously lacks. Even if I’m not able to make a working, machine-readable gcode file from a generative Machine Intelligence, I might be able to make one that is parseable enough to be rendered in a normal gcode rendering program such as the type I use all the time to create 3D print previews. If I can get a file that is merely coherent enough to be parsed by a renderer, I can take that as a meaningful output of the MI and try to manually interpret the forms it contains as a valid 3D object file using my own modelling skills.
This could theoretically be an effective workaround for the problem of extreme technical difficulty in working directly in MeshGAN space.
Moreover, it is actually much more evocative of some central, quizzical issues at the heart of MI. Can a machine be programmed to come up with programs for itself? Can a Machine Intelligence be rigorous and reliable enough in its understanding of the world of code to run a physical creative process in the real world?
One might even go in a slightly different direction and try to make text-gen python scripts. This would be an ironic/poetic endeavor, in that python has quickly become the dominant scripting language for contemporary MI systems. Could you feed a text-gen MI a few thousand very simple python scripts and train it to write its own functioning ones? Can the methods and functions of a programming language be successfully taught to an MI as the features in its latent space? For reasons that should be obvious, the name for this python-based side project will be Ouroboros.
Whew! Hope that was enlightening, if not readable! Just a few follow-up notes: I did wind up completing the gcode textgen project I came up with here, using current state-of-the-art textgen MI, GPT-2. It succeeded in creating machine-readable files with minimal editing, but sadly the files were not fully printable, since the MI did not have a full sense of the beginning, middle, and end sequencing of the code. Gcode files, created to run 3D printers, are naturally very sensitive to errors. I did manage to rig an old 3D printer of mine to go through the motions of the MI-created files! The MI files were able to run the machine’s motors without a problem, but it couldn’t have a plastic extruder attached. Still, the ‘dance’ it went through was quite nice. I mention in this piece the extreme technical difficulty of working in “MeshGAN” space, using a GAN for 3D meshes; this is a problem I have not yet surmounted, mainly because the people who publish papers about MeshGANs have, as of this writing, not yet published any code. However, for my final Thesis project I wound up meeting the Machine halfway and using a point-cloud GAN to “work directly” in 3D space. I pursued the Ouroboros project shortly, but it proved fruitless for now. Thanks for reading!
cheers,
-jeffrey