OpenAI's New AI - "Point-E" can Generate 3D models from Texts

Artificial Intelligence platforms are no longer scarce these days, as many companies pull off the AI-tech for proffering various features and channels having its own beneficiary elements. And in recent times, OpenAI – co-founded by Elon Musk, acts fully into the game. Just within a month of launch of an AI chatbot “ChatGPT”, OpenAI launches another new ‘text-to-3D’ platform called “Point-E” dedicated to generate 3D models from text prompts within minutes.

Text-to-image and text-to-video platforms has been a craze in recent days, as a bounty of companies including Meta, Google start to makes the kind of its own. OpenAI after revealing its text-to-image generator ‘Dall-E2’, now marches on ‘text-to-3D’ with Point-E.

Another Day, Another AI Platform.

Text-to-3D models – Point-E

Albeit 3D model generation is not new to the world, this Point-E approaches in a different aspect – generating discrete sets of data points in space (point clouds) that collectively depict a 3D shape. So, the models are uncountable points playing in the space forming a figure. And that’s why – “Point-E”. E referring to Efficiency, as the method can produce a 3D object within one or two minutes, which is far better than the time required by nominal way of 3D-generation that will take hours to even days.

Point clouds are synthesized easily but they do have a drawback of fineness. The created 3D-objects lack fine-grained shape or texture wherein the results won’t be perfect. To rectify it mildly, AI converts point-clouds to meshes for final result (meshes are commonly used 3D modeling & design feature that has collections of vertices, edges and faces that define an object). But is at least fast rendering with just a single graphics card.

How can you use it?

The AI firstly generates an image from the given text prompts, which is what Dall-E2 does and OpenAI leveraged that here. Then, it performs image-to-3D model, after being fed with a set of 3D-paired images paired to teach AI about the translation of image to 3D. More of like Meta’s ‘make-a-video’.

And it took several million 3D objects and associated metadata as input dataset for the AI to learn.

Unfortunately, Point-E is currently not handy enough like typing texts on a website to get the 3D model. It runs on code, available in GitHub and you need to have python installed on your system and you’ll have to be familiar in running command-line tools and programs.

Applications of Point-E – Where it can be useful?

OpenAI researchers point out that Point-E could be used as 3D prototypes to fabricate real-world objects for example through 3D printing.

Once polished of its clearness, it can be a big-thing in gaming and animation developments.

Engineers, architectural firms may leverage this simple 3D generation for their designs and sketches in minutes. And finally, wherever 3D models could be a value.