- OpenAI develops a tool to generate music from text or audio.
- Collaborates with students at the Juilliard School to annotate scores and create training data.
- Uses would range from instrumental accompaniment to music for videos and advertising.
- It will compete with Suno and Udio in a context of legal uncertainties and possible distribution models.
OpenAI advances in a tool that will allow compose music from text prompts and audio samples, according to several specialized publications. The idea is that Any user can request a melody, a specific style or a specific accompaniment and obtain a ready-to-use result..
Sources cited by The Information and other media outlets say the system could Add music to existing videos or create instrumental tracks for already recorded vocals, such as a guitar line, bass or rhythmic bases. There is no release schedule It is not yet decided whether it will arrive as an independent product or integrated into services like ChatGPT or the video app Sora.
What we know about the project
The tool in development would accept text prompts and audio inputs for generate from fragments to complete pieces according to instructionsIn practical scenarios, it is intended as an assistant that facilitates accompaniments, arrangements, and musical settings without the need for advanced training.
According to available information, OpenAI is exploring Multiple usage modes: creation from scratch, support for vocal tracks and soundtrack for videoThis approach would expand the reach of audiovisual creators, podcasters, and brands looking for customized music quickly.
Possible uses and integration
Among the applications contemplated, the sources mention the generating specific accompaniments (e.g. guitars) and composing music for clipsIn the commercial arena, advertising campaigns with customized sound and workflows connected to existing creative tools are being considered.
Another possibility is integration with OpenAI platforms: A connection to ChatGPT would simplify natural language interaction, while a connection with Sora would facilitate the editing of music tailored to AI-generated video scenes. However, this is a hypothesis without official confirmation for now.
Collaboration with Juilliard and training data
One of the most striking aspects is the collaboration with students from the Juilliard School, who would be annotating scores to provide high-quality data. This annotation includes structure, harmony, and expressive nuances, with the intention of teaching the model both musical form and intent.
Working with annotated scores can provide a more structured base than simply using audio tracks, helping the system learn progressions, dynamics, and orchestration. Furthermore, this approach aims to reduce legal risks in obtaining data and improving the consistency of the generated compositions.
Competitors and legal framework
The move would put OpenAI in direct competition with startups like Suno and Udio, as well as other actors with generative musical models (e.g., efforts by Google or ElevenLabs). The sector, meanwhile, is under scrutiny due to recent lawsuits that They question the use of protected material to train models.
In this context, messages from the head of OpenAI have emerged about the convenience of that rights holders share in the revenuesIt remains to be seen how a compensation mechanism would materialize, a key issue for the European and global music industry.
Background and technical challenges
OpenAI already tested music generation in 2020 with Jukebox., an experiment that did not materialize into a commercial product. In recent years, the company has focused on Audio models for text-to-speech and speech-to-text, and now he is returning to music with greater ambition.
The technical challenges involve maintaining a large-scale coherence in compositions, data quality and computational costTransparency regarding training and human oversight during the creative phase will also be crucial to achieving useful pieces that are accepted by musicians and producers.
If the project is successful, it could facilitate access to European and Spanish creators. Soundtracks and arrangements on demand at a lower cost, although debates over intellectual property, licensing, and traceability will persist. The industry's response will depend on clear agreements and the actual quality of the demos when they are released.
I am a technology enthusiast who has turned his "geek" interests into a profession. I have spent more than 10 years of my life using cutting-edge technology and tinkering with all kinds of programs out of pure curiosity. Now I have specialized in computer technology and video games. This is because for more than 5 years I have been writing for various websites on technology and video games, creating articles that seek to give you the information you need in a language that is understandable to everyone.
If you have any questions, my knowledge ranges from everything related to the Windows operating system as well as Android for mobile phones. And my commitment is to you, I am always willing to spend a few minutes and help you resolve any questions you may have in this internet world.