Coding for AI and ART
Once upon a time in a club in Berlin in 90s one art graduate was experimenting on lighting up the lights in various creative ways. He met a shy programmer who frequented the club, together they both experimented on how to use programming for creating beautiful art. Years later they ended up developing TerraVision, the forerunner of Google Earth. They made TerraVision out of pure curiosity to experiment how programming can make various abstract art possible.This exactly was my inspiration to use programming and AI to built art. For art I selected motion pictures. Motion pictures convey a ton of art to the audience. A group of students excited on programming, acting and direction came forward and there we go, we are making a short movie and a dance video with programming and AI.
Abstract
Students collaborate to code, write and direct a short movie and a music video with OpenCV computer vision and Google's Mediapipe AI code library in Python programming language.
Methodology
The whole project is structured with students into two subgroups namely Tech Team and Art Team and will be realised as follows:
1) Tech Team's Development of codes
The backbone of the whole project was programming. Students guided by Mr. Kevin started to gain working knowledge of Python.
A) Animated motion picture
After gaining practical knowledge of Python students were trained on computer vision to learn how computer would process images and motion pictures programmatically. They then started to develop a suit of codes which utilised OpenCV computer vision library implementation in Python. Codes were developed which used background noise removal methods and contour detection methods to convert a motion picture in RGB palette to sketch art, hereby giving an animation feel to the traditional RBG colors' based motion pictures. The codes were developed with keeping error handling to ensure the code would run without errors. The codes were then connected to smartphone cameras which would act as webcams by using a WIFI based tool webcam tool called IRIUM available on Windows, Android and Linux.
B) Dance video
Parallel to the development of motion picture codes, students started to investigate how a person dancing can be rendered to doodle art dancing. Hence after some research Google's Mediapipe Python library was utilised to make an OpenCV based Python code which detects the poses of a person dancing and then conducts holistic pose detection and renders a dancing doodle art. The code was again connected to smartphones via the method described above to record the videos.
2) Art Team's Creation and Direcion
Students with extensive background in acting, direction, choreography worked to develop a short script for the animated motion picture and the dance video.
A) Animated motion picture
A script was made by the Art team. It was about a boy who enters a forbidden cursed VR world after using a cursed VR set. Although he tries his best to escape the VR world he gets abducted by strange figures leading to the ending in cliffhanger.
B) Dance video
Students with extensive experience in dancing danced to popular Bollywood songs.
3) Debut of the videos at SSAN Annual Day 28/01/2024
Initial, limited screening of the animated motion picture and the dance video was done at the school annual day to enthusiastic positive response from the audience.
4) Release of videos on Youtube after further modifications
In future the videos will be released after undergoing further refinements to popular streaming platform Youtube.
5) Publication of a research paper
In future a research paper will be authored by Mr. Kevin Luke and the students. The paper will detail the work conducted and will show how coding and AI can be integrated at school level to enable logical and analytical thinking among the students therby leading to creating of meaningful and impactful art.
Tech Team
Sai G. 8th (assistant programmer), Shaurya C. 8th (support programmer), Akshat P. 7th ( tech support)
Art Team
Zara W. 7th (Direction, Screenplay, Cast), Akshita T. 7th (Screenplay), Natalea N. 7th (Cast, Narration), Swayam P. 7th (Cast), Aryan Y. 7th (Cast), Vaibhav S. 7th (Cast)