AI art allows game developers to be asset-rich.

The world of gaming has seen a tremendous shift in recent years, and one of the most significant changes has been the emergence of NFT games. These blockchain-based games allow players to own unique digital assets that have real value, creating an entirely new economy within the game itself. However, for developers looking to create their own NFT game or integrate NFTs into an existing title, one major challenge is generating enough high-quality assets to keep players engaged and invested in the experience.

This is where AI art comes into play, providing content creators with a powerful tool that allows them to generate an abundance of unique, stunning visual assets quickly and easily. With advances in machine learning algorithms, AI art has become increasingly sophisticated and capable of producing images that rival the work of human artists.

Two tools that have made it easier for developers to harness the power of AI art are SDXL and ComfyUI. SDXL is a powerful AI image generation tool that uses machine learning models to create high-resolution images based on text prompts. Meanwhile, ComfyUI is a ML workflow tool that feels like playing factorio for image generation.

By combining these two tools, developers can generate an unlimited number of assets for their NFT games quickly and efficiently. SDXL allows them to create unique characters, environments, items, and more based on simple text descriptions, while ComfyUI provides a streamlined workflow for integrating those assets into the game world.

Using SDXL with ComfyUI

First we must setup our environment. We used a A6000 GPU as we had it available.

  1. Download ComfyUI
  2. Download SDXL or similar checkpoint on civit.ai
  3. Setup your workflow (or find a workflow on civit.ai)
  4. Iterate your workflow to find your style
  5. Automate with python and curlconverter

Our process

Creating the assets for NEAR Monsters took several weeks of experimentation, custom workflows and rejection of thousands of not-quite-right images. We also created a way to write text in a comfyUI workflow at https://github.com/mikkel/ComfyUI-text-overlay as well as an accept-reject webapp for easy selection at https://github.com/mikkel/ImageCurator.

Here is the flow that was followed for NEAR monsters base cards:

Base monster

  1. Generate candidate
  2. Automate generation by “Copy as CURL” in the networking tab when queueing an image in comfyUI / curlconverter / python
  3. Delete bad entries
  4. Use LMM like GPT-V to name and describe the cards.

Land followed it’s own workflow.

Regardless of the technique, the result should be a CSV with the following fields:

id,name,description,rarity,types,alignment,governance_style,attack,defense,mana,url
... example row ...
0-005,Lord Malthor,A menacing demon warrior with blood-red armor.,Rare,"Demon,Warrior",Evil,"Tyrant, Fear Monger",8.0,8.0,11,https://nearmonsters.s3.us-west-004.backblazeb2.com/alpha/final/0-005.png
...

This process took several iterations and many days of active selection to arrive at a good result.

Writing on the cards

In order to write on the cards, we created a way to add text in ComfyUI then created a custom workflow. Again the process was to generate many images and then reject most of them.

You can import this file into ComfyUI to see the workflow. You will need comfyui manager as well as our ComfyUI-text-overlay plugin https://github.com/ltdrdata/ComfyUI-Manager / https://github.com/mikkel/ComfyUI-text-overlay

  1. Generate text candidates with a modification/inpainting workflow
  2. Automate the workflow by “Copy as CURL” in the networking tab when queueing an image in comfyUI / curlconverter / python
  3. For every CSV entry, call the workflow multiple times with the appropriate text.
  4. Select the best candidate
  5. Upload

This process again took several iterations and many days of active selection to arrive at a good result.

Land

Land followed a unique flow. First we generated the full form where the five cards would stake. This involved controlnets.

Then we created a shortform. This used clip_vision to match the style as well as controlnet to match the card border.

Finally we added the text using the method defined above. For the land cards we did not include mana, attack or defense.

Import these images into ComfyUI to see the workflow that created them.

Closing Thoughts

Learning new technology through game creation is fun and rewarding, as it allows us to explore our creativity while expanding our skillset. The process of designing assets for an NFT Game with SDXL and ComfyUI might seem daunting at first, but once you break down the individual steps involved in creating each asset type, things become much easier to manage.

Remember to give yourself time to this right. This process took several weeks of experimentation and selection. You are defining the heart and soul of your game through these assets. The rest of the game should unfold from how these look and feel.