July 21, 2021
Section 1 of our three-part piece required an audio remix to make changes to the placement of the Mandarin voice part, and to address some miscellaneous sonic issues. The image below is a representation of the audio parts in Section 1.
In this image, the orange clips at the top are Yunfei's voice part. Meeting over Discord, Yunfei and Jeremy adjusted the placement of the voice part.
The darker orange clip is Yunfei's bell piece, which also includes Tang's contribution imbedded within in. Ideally, to have the best mixing flexibility, we would want Tang's piece to be a separate track. Unfortunately, during a recent computer issue, Tang lost access to his Ableton session file. We no longer have access to the original mix of his part before it was joined with Yunfei's part. This underscores the importance of backups when working in this medium.
In our previous checkpoint, our piece had reached the third major iteration (T3). The latest iteration is number 6 or (T6) from 7/12/21:
The scene layout, in terms of authorship of the pieces, remains the same:
Section 1
Primary composer: Yunfei
Additional parts: Tang/Jeremy
Section 2
Primary composer: Jeremy
Additional parts: Yunfei
Section 3
Primary composer: Tang
Compared to the previous version, which used two of Tang's pieces faded together, we are now using only his piece named “Lost Letters” as the final section.
For the most part, the composers feel that the music is done, and we are now spending more time working on the visual concept. We've added a transition between the sections using Yunfei's original ocean video and sound, mixed with some of Jeremy's animation. Another big change is that we're now working with more varied movement footage from Sammy for each section. This footage was captured on her cell phone.
It's still possible that we will change the video processing using some other techniques. For examples of some of these, see below:
Each scene is rendered in real time using a custom Touchdesigner component that Jeremy has built. This process is part of the creative act, incorporating many elements of intentional randomness, and does not follow a traditional video editing timeline. Below is a table that is used to align the scenes with the music based on time marks.
Rendering our 8 minute piece takes two or three times that amount of time, even for 1280 x 720 video. Visualizing edits can be time consuming. All video is rendered on the fly, including audio-reactive elements which scroll through the video clips within each section.
Challenges
In addition to some of the technical challenges mentioned above, all of the members of the NS Blue Team have had other challenges over the past few weeks that have impacted meeting times and creative work.
In the US, an intense heat wave has impacted many areas. The heat wave has strained climate control in Jeremy's studio, forcing work to be done in the early morning or later at night. Sometimes only an hour or two of workable time is available. Significant noise pollution during the summer lawn mowing season further curtails the time available to concentrate on audio work.
Yunfei has been attending an online festival with lots of workshops, and she is preparing for a move later in July. Sammy started a new job doing animal rehab at SeaWorld, and has also relocated temporarily.
In the past few weeks, Tang lost a beloved pet. Unlike the US where things are opening up for the most part, Tang is still experiencing COVID lockdowns in Thailand.
June 18, 2021
As the team entered the next segment of the “Fragile States” writing prompt, we first discussed whether or not we wanted to continue along the same lines with a different movement artist, or change directions completely. Up to that point, our entire creative approach was based on having a movement artist as part of the team – rather than just focusing on music/video.
Everyone agreed that we'd lose more time/energy by changing directions, so we discussed possibilities with the NS core team. Out of a number of great people to chose from, we welcomed Sammy Gerraty on May 5, 2021.
From that point the process of iterating video and music continued, and we set a regular meeting time of Wednesday 11 AM EST. This helped us pick up some of the momentum we had lost during the transition. Sammy supplied some rough movement footage in response to some of the current music, and Jeremy continued with test renders using that footage with creative effects.
The environmental theme expanded beyond just oceans, and started to incorporate other ideas such as nature trails, with the idea that all footage we use would be footage shot by the group. Another element – noise pollution – began to creep its way into the project (conceptually and literally) as the area where Jeremy lives outside of Philadelphia has intense noise pollution at times.
We iterated further on the music with Tang adding elements to Yunfei's piece, Jeremy adding elements to Tang's piece, and Yunfei adding elements to Jeremy's piece. Having three strong composers in the group means that each is set up with their own method of composing, both in style and in terms of technical capability. Tang and Yunfei both use different versions of Ableton Live on a laptop but without any professional audio interface or monitoring environment (mostly earbuds).
All tracks eventually funnel to Jeremy where he mixes as needed using a professional audio interface and monitors for best results. Jeremy uses Ableton as well, but for this project he chose to use Propellerhead Reason as a personal preference.
The Blue Team piece (currently untitled) consists of three main sections:
Section 1: Primary composer: Yunfei with additional parts: Tang/Jeremy
Section 2: Primary composer: Jeremy, with additional parts: Yunfei
Section 3: Primary composer: Tang, with drone guitar by Jeremy
This brings us to the outline of the piece as it currently stands.
current outline
Each section of the music has its own project file where that section is mixed and mastered. The resulting audio file is imported to a combined project file where the three sections are arranged for the final mix. Below is an example of the current state of the combined mix:
About the current video draft:
The split screen is intended as demo to show both the nature footage and some processing on Sammy's movement footage. For now these elements are shown separately, but may get merged/processed together. This rendition uses Tang's piece “Lost” (with a bit of drone guitar from Jeremy) transitioned into Tang's “Lost Letters” piece. We weren't sure if we wanted to include both, but this was a draft to see how it would work.
The plan from here is to get more ocean footage, preferably from the team. If not, we can make the decision to use some stock footage. We also have more nature trail and stream footage for section 2.
Note: This video is not being made with a typical timeline editor like iMovie or Resolve. It uses a series of modules I developed to get detailed control of playback time, effects, etc. On the one hand, it opens up a lot of capability, on the other it can be time consuming to make changes. So detailed feedback on the direction we want to take is essential. And since everything is really driven by the music, we will still count that as the first element to finish.
EQ Touchup in Adobe Audition
In many cases, listening with earbuds or laptop speakers doesn't allow one to hear certain audio issues. In this case, there was some very high pitched material in one of Yunfei's rough mixes. These are shown by the prominent orange horizontal lines between 10k and 12k in the frequency plot. In some cases (such as noise music), this might have been intentional program content, but in this case these artifacts would be too piercing and distract from the piece, so they were removed with EQ.
A peek into the files needed to create even a short piece of content
May 1, 2021
The Nomadic Soundsters Blue Team started with three composers (Yunfei Li, Supakorn Wongsumdang (aka Tang), Jeremy dePrisco) and movement artist Alex Oliva.
During our initial meetings we did not take many notes, but we brainstormed some approaches and landed on an environmental spin for the writing prompt of “Fragile States”. Eventually, we also discussed incorporating spoken word, via some poems relating to the environment.
In the weeks that followed, each of the composers began creating their own musical fragments to share with the group and seek feedback. Due to technological issues and spacial/time zone scheduling, it was impossible for us to schedule synchronous writing sessions. Each member, including Alex, also made some voice recordings of the quotes that everyone selected from a wide range of authors.
Our approach to movement was perhaps the most challenging, since the composers had little experience working with that medium. Alex guided us on some ways to work together and by looking at samples of his past work we brainstormed about possible approaches. For a time, there was a lot of discussion about tying in some sensor data via Arduino and other methods, but this was eventually abandoned due to the amount of time it would take to develop for such a short piece.
Yunfei and Jeremy provided some nature footage as placeholders for the concept, and Alex provided some movement synchronized to the rough draft of the music as it unfolded. The intent was to get better footage later with a clean background, but this was enough for Jeremy to begin compiling some proof of concept recordings using Touchdesigner to manipulate the footage using filters and video processing techniques.
Coming up on the first checkpoint, we knew we wanted to have a 5-6 min piece that alternated between movement and nature footage. What is shown in our first checkpoint draft video is purely conceptual to show some of the work that was done. At this point, musical elements by Yunfei and Jeremy began to take shape, but we had not landed on what elements Tang would add (though there were some fragments proposed). As we will see in future updates, the musical collaboration is based more on asynchronous overdubs and adding layers rather than synchronous composition or jamming on musical ideas. This can be a challenge for everyone as it requires extra time and energy to communicate the intent of the various fragments, and how we see them fitting into the overall piece.
However, we also had another major challenge. Alex had a job opportunity that he couldn't pass up and had to drop out of the NS collaboration. Going into the next phase, we would have to decide if we wanted to find another movement artist and, if so, how we wanted to proceed.