INDIVIDUAL PROJECT

AI Composer for Live streaming

WHAT'S
MUSIGO?

Musigo is an AI music composer that generates real time background music for live streaming videos based on the user’s preferences.

EXPLORATION

Before starting this project. I was curious about how AI is trained to interpret the world. To understand it, I explored many Runway models and existing AI tools.

OPPORTUNITY

AI can generate real-time music for live streaming videos by analyzing the video contents in real time, based on the training of the video database.

How to train the machine to understand user’s expectations?

To explore how to train the machine to understand users’ expectations, I prototyped the algorithm by playing different genres of music and asked users about their feelings and imagination. Then the participants checked the adjective words and context music they felt connected with the music.

Insights

People have shared opinions and feelings about music.

They also have their own preferences for using the music.

PROTOTYPING AI ALGORITHM

DESIGN CHOICES

SYSTEM DIAGRAM

The machine can generate music based on common public preferences.

Allowing users to tune the generated music.

Deep learning of users’ unique preferences and adjusting the algorithm.

USER SCENARIO

Michael loves traveling, he also likes sharing his experience on live streaming apps. He sees a new plug-in called Musigo on Instagram live. It says that he can use AI to make music for his live. He wants to try it.

Key user points

Having royalty-free music along with the live video that matches well.

ITERATION

I want the design to be generative, integrated, and personalized. After many rounds of iteration. I decided to design a minimal style UI that can be easily embedded into an existing live streaming program.

FINAL FEATURES

Please turn on the audio for the best result

GENERATING AI MUSIC

Musigo can generate AI music from what it’s seeing through the camera. Computer vision and real time data detects objects and motion, and monitors weather and time.

DEEP UNDERSTANDING OF USER'S MUSIC PREFERENCE

Musigo can generate different styles of songs until the user finds one they like. The system will learn from the user's behavior, so that it can anticipate the user's preferences.

ADJUST THE MUSIC BY HUMMING A TUNE

User can adjust the melody of the music by humming a tune.

EMPHASIZING HUMAN VOICE

When user starts talking during live streaming, the music will be lighter, and the sound volume will be lower, so that the audience can hear the voice better.

PANNING

When the user is panning the camera, the audio levels automatically adjust based on the direction the camera is facing.

Thank you for viewing!

MORE WORKS