An Essay
The impact of AI is being talked about everywhere these days. Chances are this is one of several articles passing by your scrolling that mentions it and the impact it has on everything we touch. One of the major topics we see is Generative AI’s impact on “art” in all of its forms, from image to music and dialogue. It has given the ability for the average person to spring ideas to reality while giving rise to the notion the creator can and should be replaced. For a moment though let’s view it through a slightly different tinted lens, not of AI but of technology.
The best way to look through a lens is to look through it at the past. During my middle and high school years I survived classes through drawing. While my grades were less than stellar because of this it was a mostly non-disruptive activity. Mostly, since people collectively find fascination in watching something be created from nothing. Also being a young teenage male before the internet’s more adult elements became readily accessible, the sketches not always school approved material. Like minded people tend to form groups and I was no different having several friends who were talented artists in different mediums. One friend, Lee, was like me in that he enjoyed drawing, however his skills far surpassed mine as he had an amazing talent I could not grasp. He had the ability to bring color to his art, and he could do it in several mediums from pencil to pastels and paint. It would inspire an emotion that I would wrestle with identifying for years to come and I suspect most people have today but can’t see it. It subtly drives the sentiment that the person is replaceable.
Enter technology. In 2000 our high school brought in a set of high DPI scanners, full range color printers, Macs, and most importantly: Photoshop. Now this before the digital art renaissance and looking back through this lens, the technology was very limited compared to what we have today. What it provided, as I saw at the time, was a level playing field. The tools gave me the ability to shade evenly and color cleanly in the lines conquering my inadequacies and understanding of brush order, even consistent stroke pressure, how to layer color, or even the basic of fine motor control to stay in the lines. I had been handed the tools but none of the knowledge or experiences to make proper use. It didn’t matter to me, mostly because I didn’t understand the deficiency in myself, I was able to take my drawings and give them the life that Lee could and by god my arrogance was going to let me show it off. The tools let me create work that he could do so easily and naturally. While I had joy and pride in what I was now able to do, what I produced was absolute garbage. As technology oft does, it provides easier tools to use but none of the knowledge and experience to produce good outcomes. My friend was a good sport about this and could have responded with expected insults to the art and myself, but he didn’t. Instead, he was far more mature than any teenage highschooler has a right to be. He pointed out the flaws in color selection, how shadows in color are different than in line work, where highlights should exist, and gradients could give depth. We would both go on to learn more in the digital medium and how it could be used to augment our work. More importantly, it helped our art do what it is intended to do.
Art is done to help form human connections.
Time and technology would march along, I slowly moved from sketching and drawing on paper to more digital focused works such as 3D asset creation, texture designs, and model implementation for video games. The tools would continue to change, evolve and the outcomes that could be produced are not always the best off the bat. However, it was still enjoyable to see when someone connected with your work, sometimes in the manner intended in your own head, and other times in completely unexpected ways.
Fast forward to 2022 and I have a career in the HPC industry designing systems and applications for rationalizing and reacting faster to generate better outcomes with each iteration. We could have an entire article on the development and evolution of heuristic analytics systems to modern AI however we will leave that for another day. At the time, I still enjoyed drawing. Working with some colleagues at Ohio Super Computing Center I was introduced to a viable version of Stable Diffusion and there’s a wonderful collision of tech and art brain. While it was not very good at generating action frames or anatomically correct people, it could generate complex repeating tile able patterns and backgrounds, something I had normally struggled with and if I am honest, I didn’t enjoy doing. I could also use it with embedding and generating color and fabric patterns on the fly to help come up with detail ideas for sketches.
Stable Diffusion also opened a concept I had not previously worked with, Low-Rank Adaptation or LoRA. While I had been working with general model fine tuning up until then it was a full sweeping approach for very specific outcomes. LoRA allowed a targeted approach and could be done at a smaller scale with fewer resources alleviating my need to beg and borrow reservation time. Thus, I had a very bad idea. Use this new tool and my technology background to create models to help make “my” style more consistent.
There were a lot of challenges and learnings that would go into this, along with a few $600+ month power bills that required apologies to my wife. A follow up article on the technical process that went into first trying to teach a model to tell me who I am and then teaching a model the styles that influenced my work the most might be in order. My work was heavily influenced by Katsuaki Nakamura (Visions of Escaflowne) and Hayao Miyazaki (Studio Ghibli). That’s right, we can’t have a Generative AI conversation without reference to Hayao Miyazaki. While his style is unique, I believe why most western audiences gravitated to his work is the amazing stories he told and the emotions those stories invoked.
I was working on a project for a friend doing a custom image profile and avatar for them as I completed the LoRA fine tuning. This would be a good test to run a sketch through as an embedded image and have the tuned model take my sketch closer to what I envisioned between Miyazaki and Nakamura styles that I found myself leaning into. From a technical standpoint, it worked amazingly well and performed in an optimized manner. Color selection, shading, line width and weight, and the fine line between too little detail and lost in the details.
As art though, I felt nothing.
Anyone who has created anything has their highs and lows. Some projects you complete and look at with pride and satisfaction at what has been created. Other projects don’t go well and maybe you finish them but don’t like them or maybe disgusted with them. Some you dislike and never complete leaving them half finished to be forgotten. In each of these cases you form a connection with the thing you have made. Art helps form human connections, even with yourself. For the amazing technical accomplishment I had achieved, I felt no joy, frustration, sorrow, disgust, love or hate for it. I looked at the completed thing and felt nothing.
I stopped drawing after that. Not because I hated it or was frustrated with the work or disgusted with myself or any of the cliché responses a drama story would say I should have gone though. I simply felt nothing from drawing after that, so I stopped. I continued other creative activities such as painting models and CAD design for 3D printing. What this experience provided was personal context and understanding for where this technology and its impact was headed. It would force me to also look back at other technological changes to the creative field to recognize and acknowledge its emotional impact.
Today we do not need to build a dedicated environment for image generation such as Stable Diffusion. The average user doesn’t need to fine tune models for specific styles of art as they are understood by the underlying model and can generate “fast enough” and “good enough”. Miyazaki became a victim of his own memorable storytelling and ability to have his art connect with audience. Much like his influence on my own style, to a wider audience who want to create something but otherwise didn’t have the talent to do it, they asked a prompt to create an image in a style that they could remember a connection with. People have the desire to create and connect, and the tools now allow more people to do the former, but unfortunately not the latter. Connections between people still come from people. What we are left with is fun images being generated but otherwise leaving both the creator and consumer mostly emotionally empty. Internet success snowballs and the original creations in “Ghibli Style” were quickly replaced by memes in “Ghibli Style” taking Generative AI and evolving it into Regurgitative AI.
An interesting side effect of creating tools that allow more people to participate in something is the interactions that then open between people who would otherwise not have been able to connect. I run a gaming and development community called T3stN3t and we have an area set aside specifically for sharing art. Normally it’s the artistic part of the community who shares and interacts in this area though all are welcome to participate assuming it remains respectful. Earlier in the year one of our members, who normally only comments on in progress work, jumped in to participate. The conversation was around vomit inducing cute things vs horror inducing nightmares and how close those two can be. It turns out that most creatives are probably not entirely correct upstairs. He joined in the exchange with some ChatGPT generated images they had created, and the feedback, conversation and creativity loop continued, with a new member coming from a new direction with a different set of tools. A few days later he snapped and blew up at the community.
Look, the result of 400,000,000 stolen images and now with a keyboard I can do what your able to do. Don’t be helpful.
Be angry!
This **** posting is replacing you.
The specter of that emotion that always lurks around in the background given voice, shape and form. Much like high school me deep down wanted to lash out at Lee and hurt him by showing I could be just as good without the talent, this 20 something denizen of the internet was lashing out at us. People have a natural desire to create, and when they are unable to but surrounded by those that can, manifest their frustration as destruction. I believe this is further pushed when you finally have the tools to give life to your ideas but once sprung to “life” you feel nothing towards the creation. When left with the void something will inevitably try to fill it, negativity is readily available.
This leads us to the crossroads we are at today. It’s easy to drop into a prompt and ask for a random image in some anime style. Why anime and manga related leads the pack is likely an entire case study in storytelling and culture impact but it is clear that the medium has deep connections between the artist and the consumer. Beyond Hayao Miyazaki’s incredible depth of works the medium is full of memorable and impactful stories such as Yoshitoki Ōima “A Silent Voice”, Naoshi Arakawa’s “Your Lie in April, or Makoto Shinkai’s “You Name” and so many more. Each of these goes beyond the art of the images and tells unforgettable, emotional, and human stories that connect with the reader or viewer.
The toolset available to the creator across industries is rapidly evolving. Tools led to opportunities to push past boundaries that might have once seemed frustratingly impossible to overcome. The invention of the chainsaw gave the lumberjack the ability to more efficiently clear an area and the average person the ability to clear a fallen branch in their own lawn. AI provides tool optimization for experts and wider availability to the amateur. There will be unfortunate cases where tool optimization replaces the human element, such as on things that had very little human connection like advert creation on high volume thin margin consumer goods. However, art is to help form human connections. When we hear the sentiment that art can be solely AI generated, the artist is not needed, is the person stating this speaking from ignorance or are they wrestling with the subtle and quite voice deep inside themselves.
When someone says, “You can be replaced with AI” I ask, “What Lie can AI tell in April”?
-Bob Krause