How to Create a Virtual Choir in 360 Degree Video and Ambisonic Audio using the Acapella App and Adobe Premiere Pro

So you want to create a Virtual Choir Video … In 360 Immersive Virtual Reality? With ambisonic, directional, adaptive audio? Prepare to work your ever-loving ass to the bone, but it can absolutely be done – and it can be an enriching, collaborative, rewarding process. The best part about doing your virtual choir in a 360 environment is that it gives you a larger canvas to work with. If you have a large choir or orchestra, this is a HUGE advantage. I personally hate those tiny little squares where you can barely see the expressions on people’s faces. With 360 video, rather than just a little 16:9 frame, you can put your singers and instrumentalists all the way around the viewer – and instead of choosing what your viewer will see, you can allow them to “choose their own adventure” and create their own “camera angles.” If they want to obsess over one particular performer, they can totally do that! I’ll do my best to walk you through all of the stages of it using my experience creating this video:

This is a long and involved process, but let’s do this together. When the Pandemic broke out and everybody in my choir was forwarding links to me of other choirs doing virtual recordings, I swore I would never be one of them.  My heels were dug in and Eric Whitacre was on my shit list.  But as 2020 headed into fall with no end in sight, I finally gave in.  I am assuming that you have a basic working knowledge of Adobe Premiere Professional – and I’ll bet I have missed some important pieces along the way. Feel free to suggest changes to me in the comments below this article. You can also do what I did: Use what you know and Google your questions. Somewhere, somebody else has stayed up until midnight writing a blog post like this one that will give you exactly the answer you are looking for!

Stage 1 is your planning stage. Assemble your team – find hard-working collaborators who are willing to pull their share of the load and have a penchant for detail. Have a fun organizational meeting where you will outline the entire process and set realistic expectations regarding the scope and work involved with your project. Set clear expectations of what each person will do including a production schedule with due dates for each element. You can’t do it alone, so make this about collaborating and connecting through artistic work. Here are some ideas for key positions:

• Fund Raising Manager and Committee
• Publicity & Social Media Campaign Manager and Committee
• Choral & Orchestra Managers
• Recording Scheduler Manager & Team
• Technical Manager & Team
• Artistic Director and / or Music Director

For our video, we chose to do our annual production of Part 1 of Handel’s “Messiah” plus the “Hallelujah” Chorus and “Worthy Is The Lamb – Amen” for Choir and Chamber Orchestra. A note about repertoire choices: Pieces with fewer tempo changes are easier to carry out. Generally, “Messiah” proceeds straight through with relatively few tempo changes … except for the Arias and Recitatives – but this is where you get to use your creativity and ingenuity to do some problem solving. For the choral numbers, I created conducting videos out in nature without any kind of audio reference. But for the solo movements, I scheduled the soloists for a recording session with only the “continuo” players present, socially distanced with air purifiers between them. This presented another set of problems – when there are whole measures of silence because the players are not there – but I have to continue conducting and the soloists had to pull their note from thin air! Another issue is dynamics. Ideas like “loud” and “soft” are a very comparative concept, and when your singers are performing without the visual and audio cues to which they are accustomed, they will tend to perform everything at a single volume. Of course, you can always change their volume in post production, but I would encourage you to stay away from altering the volume and pitch in post production because it de-humanizes your final product and gives your video a mechanical / robotic quality. I can hear autotune a mile away and it is a major turn off.

Assemble your starting materials. Here is what you will need:

• Your music – choose your edition, and reference recordings if they exist and make a detailed plan of all your artistic choices regarding where to breathe, phrasing, dynamics, bowing marks, articulations, etc. I *do* believe in buying originals, so if you have a 40-voice choir, buy 40 originals, and although it is illegal, I have heard of people who mark up a copy with all of their artistic choices, scan this copy and then make this available to their choristers and orchestra. This work will result in a higher quality end product and make everything far less frustrating and save rehearsal time and endless e-mails with complicated instructions. I don’t do this, but I can see why other people might do this.
• Starting (practice) videos. Film your director conducting the entire work. Leave at least 10 seconds of silence at the beginning of each movement. Your performers will need time to get situated before they begin to record and they need to say their names before the music begins so that you know which audio goes to which video. Get as close as possible and use a contrasting background and good front lighting. Indicate a very clear tempo, entrances and cut-offs, dynamics, articulation, etc. This is hard to do without an audio reference. Expect to record 30 to 40 takes. If you can assemble a bare-bones group to record safely, you will find that to be super helpful reference. You will need a separate camera and microphone for each person and the audio should be recorded to a multitrack recorder so that the mix can be customized in post production. You will then go through your footage and select the best conducting videos. Look for consistent tempo, clear information, and, of course, no skipped beats or measures! Once you have selected your conducting video, import that video into your editing software (I used Adobe Premiere Professional.) Then, while watching the conductor in the video, use a microphone and record a reference “clap” track. Subdivisions are usually necessary – especially for slower tempi. Next, hook up a keyboard to your computer and record a reference track for each of the vocal parts (separately – not on a single track.) It will be virtually impossible to get these flawlessly aligned, but the clap track will help immensely. Try to include your dynamics, attacks, cutoffs, and as many stylistic elements as possible, since the singers are likely to mimic what they hear and if everything in the reference track is at the same volume, they are liable to sing similarly. If you can have a singer with an excellent vocal quality record their part with the piano, so much better. A great vocal model will help your weaker singers enormously. Conversely, a hasty, careless rendition could doom your project. Then render out a video for each of the individual movements for each of the vocal and orchestra groups. For example, when you render out the video for the tenor 1 group, you will want to include all of the vocal parts quietly – in the background – or perhaps only in the left or right track, but you will want to boost the tenor piano track so that it is clearly predominant. They shouldn’t need to “fish” for their pitch. Then upload these videos to a sharing platform like Google Drive, DropBox, a WordPress website, etc. and share them out to your choir so that they can start practicing on their own. My orchestra requested that the harpsichordist record her part first as a pitch reference. I thought that was a great idea.
• It is never too early to begin fund-raising and building your advertising campaign. Don’t wait until you have a final project to start talking it up. Communicate your excitement and your vision to create a “buzz” from the very beginning. Talk up the project in a way that makes everybody want to participate and creates an air of excitement for the release of your video. The more people you involve at each stage, the more successful your project will become. Give everybody something to do and share ownership. Encourage every contribution. Give people a sense of purpose and meaning. While your artistic production team is busy creating starting videos and editing scores, your advertising team should be recording them, interviewing them, and sharing videos to Facebook and connecting people and advertisers to your project.
• Create “how to” videos that explain how the technology works. Recommend software and hardware that your musicians will need in order to collaborate. Some people will want you to tell them what to buy and how to operate it: step-by-step like a recipe. Other people will just want to know the system requirements so that they can research and buy their own equipment (or see if equipment they already own will work.) Share this with your collaborators as soon as possible so that they can make the necessary purchases, hook them up and practice with them. We used an iPhone app, “Acapaella” from “PicPlayPost.” It is the best app I have found for collaborating, but it limits you to Apple devices. For this app, you need an iPhone or iPad and a microphone that has a built-in headphone jack. I personally found the Shure microphones to be the easiest to set up and use, but you can use virtually any USB microphone if you get the lightning camera adapter from Apple. You can buy 3rd party adapters, but I have heard that they tend to introduce compatibility issues and drop errors that will put your singers off. I especially liked the camera adapter that allows you to plug in your power cord to keep your phone or tablet charged while you work. My personal favorite setup was to use a USB mixer hooked to my phone using the camera adapter. You plug the mixer into the phone using the camera adapter and then you can plug in your microphones and headphones into the mixer. It allows you to record in stereo and to use higher quality microphones, but it can also introduce random latency issues and some people somehow managed to end up recording the reference track along with their audio and that could be tricky to diagnose. Headphones are also important in your selection. Cheap headphones can leak sound into your recording environment, which then gets picked up by the microphone and you’ll hear whatever is in your reference video and it will ruin your final product. The best headphones will be noise isolating and visually unobtrusive. You may be able to set up a recording station(s) where people can schedule time to use equipment that is set up and monitored by your team. This is great for people who have a tough time with technology or who can’t afford the technology, or who don’t have iPhones or iPads. But it is a bit tricky to schedule.
• You will want to create a shareable (Google) spreadsheet to track your participants. Who is recording (with their contact information). What movements each person has completed. Who has each track at any given time. You will also want to create a sheet that includes your production schedule and when each part is due. Breaking down an enormous project like this into smaller, achievable goals helps people get organized and stay involved. There are a lot of moving pieces and this is an easy way to keep everybody up-to-date. You probably don’t want to give everybody editing access, but letting them see the information can be very helpful.

The next stage of the project is when your choir and orchestra begin recording their work. The Acapella app allows your people to record in teams of 8 (it actually allows 9 participants, but one block will be taken up with your reference video.) Every virtual choir involves some version of combining the videos of many participants. What most choirs do is have each person sing along with the reference video and then the editor lines up all of the individual videos. This has the advantage of making it possible for virtually anybody to contribute to your project from virtually any smart phone. It has the disadvantages that your singers and instrumentalists cannot hear each other and results in having to line up a ton of videos of different lengths. But if you decide to go this route, most people will create some kind of reference at the beginning and end of their videos – usually a series of claps. So, for example, the reference video may ask you to clap with them a series of 8 claps in tempo at the beginning and end so that the editor can synchronize the videos for final production relatively easily. Using the Acapella app avoids this step. Every video that is submitted will already be synchronized and as people record, they will have the option to listen to each other. Singers who don’t read music and / or don’t have a strong music background will feel much more enabled when they can sing along with a few of their fellow singers from their section.

Pro tips for the acapella app: An annual subscription is much cheaper than paying by the month. If you have the pro subscription, you can record up to 10 minute clips, it removes the watermark, and allows you to record in 256k audio (instead of 128k audio). It’s well worth it. Start by choosing how many participants you will have in each team (can be up to 8 people). Always choose the maximum length (10 minutes) because the app will shorten the length to match your reference video automatically – but if you choose a length shorter than your reference video, it won’t lengthen it. Better safe than sorry. If you have movements that are longer than 10 minutes, I don’t think you can use this app. For my video, the “Worthy Is The Lamb” ended up being *exactly* 10 minutes long. I could have (probably should have) separated out the “Amen.” Once you have selected the number of participants and layout, you can’t change it later, so choose carefully and plan it out. Consider carefully the order in which everybody should record and create and share a chart of each project so that each person will know in which square to record. For this step, you will need to know exactly who is going to participate. You can ultimately trim your video to get rid of unused squares or fill the unused squares with photos and info pieces, but the better you plan, the better your final project. Your section leaders will likely give a better vocal example for other people to follow, so consider asking them to record each track first – and listen to their recordings and ask yourself if this is what you want the other singers to sound like. Your concert master should record their part before the other first violins, etc. Upload your reference video – and make sure that when you render your reference video they have the identical length and number of frames. I used Google Drive a lot for moving files to and from my iPhone and uploading them for the Acapella app. I would put the reference videos into a Google drive folder, then upload them to the files on my iPhone, then transfer them to my photos, then upload them into the app as a starting video. When you upload the starting video, make sure that the app marks the *actual* end of the video. Sometimes the app would randomly cut off time from my reference video – so verify in the upload screen that the video is the correct length. It’s tedious, but you will be sorry if you don’t double check when people call you because their video cut them off before the song was over. Once the video is uploaded, it’s time to save and share. The person doing this will want a fast phone and a fast internet connection, because they will spend a LOT of time saving and sharing and rendering. You will select “save and share.” You will also want to include a set of expectations for your singers that include 1) Dress code 2) Background instructions – what should be in the background of the video – I asked my singers to adhere to the dress code and show their personality with their backgrounds. 3) Photos of how you might set up your phone, your microphone, and your music stand so that your phone can see your face, your microphone is in close proximity (the further away the microphone is, the more background noise you will hear) and whether you want to see the music, music folder, and/or music stand in the picture. 4) lighting (you don’t want to stand in front of a window that has sunlight coming in, because the strong backlight will make your face too dark to see. Rather, face the window so that you get the sunlight on your face. Etc. 5) Listen for background noise – construction noise, appliances, doors opening and closing, people talking, etc. 6) How should the video be framed – do you want whole body shots, just the singer’s face, the face and shoulders, etc. – give them an example photo. 7) What level of quality do you expect? It’s important to set expectations regarding how much perfection is required – as it is, you will get a wide variety of submissions, but you can minimize this through setting clear boundaries. For example, some people will become increasingly perfectionistic and take 40-100 takes and then send you long e-mails about how they took too loud of a breath on beat 2 of measure 39. Others will do one lousy take and call it good. Trust me, you don’t want to leave this up to chance. Important tip: As people record their parts, you will want to share and record them in order. If you share the same project with 2 people at the same time, you will create 2 timelines for that project that will then have to be edited and combined in post production. It defeats the point of using the acapalla app. I used a spreadsheet to track who has each project and who needs to get it next. As each person came up for their term, I would click “save and share” and then select “collaboration” and then either “SMS” or “e-mail”. When they would share it back to me, I would usually then save it to my camera roll as a private project – and then I would delete it from my camera roll, because the point wasn’t to get it on my phone, the point was to upload it to my profile so that I could then access each project at each stage of the process. It was time consuming, but it saved me on more than one occasion when somebody would record over somebody else’s video and I didn’t discover the error until I was in post production. Then I went back, pulled up the most recent version that still had that person and trimmed their video and put them back into their square. Here are the generic step-by-step directions that I sent out to everybody who participated.

• Turn off all notifications on all your devices and silence your ringers before you start this process.
• The microphone should come with a standard USB cable.  This should plug into the apple branded USB camera adapter.  If the microphone comes with a lightning adapter, you can plug it in directly. If the microphone comes with a “USB-C” adapter, you will need to get a USB-C to USB-A adapter in order to plug your microphone into the Apple USB Camera Adapter.
• The power cord for your phone should also plug into the female lightening port on the USB camera adapter.  This is actually very important, because the microphone will otherwise drain the battery on your iPhone rather quickly and it will be a short recording session! Be sure to use the power adapter that came with your phone (or equivalent) because if you use an adapter that is not rated for your phone or tablet, you will get a message that there is not adequate power.
• The USB camera adapter then plugs into your iPhone via your phone’s lightening port.
• The noise isolating headphones should be plugged into the headphone jack on your microphone (not on your phone – this is only possible on the older iPhone 6 – but you’d be amazed at how many people still have and use them).
• Your microphone may have several options from which to choose including stereo, mono, cardiod, etc. You will probably want to choose “mono” for this production, since this will be easiest to manipulate to create the ambisonic audio in post production.
• Plug everything in before opening the app or clicking the links. Consider closing all the open programs on your phone before you start recording. You may be prompted to download an app or install software when you plug in your microphone – but it is very unlikely that you will *actually* need to do this – just ignore it the notification.
• A very important note:  When you click on one of the links that I will send you and you choose to open the project on your phone, it will replace any project that you were previously working on – deleting it with no chance of recovery.  I’ll go through the process of saving the project later, but for now, just know that if you open a new project without saving any existing project, that project will be lost.
• When I send you the link, it will show up as a video that you can play and below that will be the words, “Acapella mixcord”. Rather than clicking on the video, click on the words below the video.  At this point, (if you have already downloaded the app, paid for your membership and created your profile) your phone should open the acapella app.
• Click on “Collab”.  It will then pop up “New Project.”  Click “YES” to continue (assuming you have previously saved your work)
• It will then import the project to your phone, replacing any existing projects.
• When it finishes the import, you will come to a screen that includes “cancel” (to go back to the main acapella menu) “save & share” (that I will go over later) and at the bottom of the screen are a play button (to play the project as it currently stands) and a speaker button.  If you click on the speaker icon, it will take you to a page where you can make adjustments to the sound of any of the squares in which there is already a recording.  You can adjust the sound of any of the squares by first clicking on the square so that it is given a red outline and then making whatever adjustments you would choose in the options below.  I would ask that you not make any adjustments to the audio except for the first slider, which is the volume.  Feel free to adjust the volume any way you like for any square so that it matches your preferences of what you wants to listen to while you record.  Any changes you make will show up when you go to record.  When you have finished making adjustments, click the green check mark at the top right of the screen.
• Feel free to explore the other icons on the bottom, but none of the others are relevant to this recording.
• When you are ready to record, click on the square that you will be recording into.  Please refer to the reference PDF that I sent you so that you know in which square to record. If you click on somebody else’s square, you will record over them. Please don’t do that!
• This takes you to the recording screen.  You *can* use either of your phone’s cameras, but we need to use the selfie camera so that you can watch me conducting (even if you don’t want to.)  
• There is an icon of a microphone that should be set to “256 kbps stereo.”  The microphone should be highlighted in red. (If it says 128 kbps and you can’t change it, then you need to get the paid version. Exit the recording and make your purchase and then return.)
• There is an icon for a metronome.  You will want to click on this and turn the metronome off.
• There is an icon of a jack.  You will want to click on this and sing or play your loudest notes so that you can adjust the gain.  You will be able to adjust the gain with the slider on top as well as (possibly) with a knob on the microphone itself.  There are other adjustments, and you can decide if you want to mess with any of them.  I usually leave them at their default settings. Pro tip: When you adjust your gain, too soft is better than too loud. If a recording is too loud, it will be distorted and clipped and it will sound bad and this cannot be fixed. If it is too soft, it can always be turned up in post production.
• When you have made all of your adjustments, click the x in the upper left corner.
• Please refer to my notes regarding how to frame the video that you are about to record, what to wear, what kind of background, what kind of lighting, etc.
• You should now be ready to record.  
• Press the red dot to start and stop the recording. Please say your name clearly into the microphone at the start of the recording before the music begins – this helps me link the correct audio track with the correct video track.  Every time you stop, you will have the option to “redo” or “continue.”  This is self-explanatory.  “Redo” takes you back to the beginning of the track to start over. “Continue” takes you to the next step in the process.
• Please DO NOT TAP YOUR FOOT! The microphone is likely to pick this up and transmit the sound, creating some very unpleasant percussion. I had to mute a singer who tapped their toe – and if you listen with good headphones, during the “Hallelujah” chorus, you will discover I missed a track where a singer was tapping her foot. What a bummer.
• NOTE:  When you complete a track, allow the video to record all the way to the end.  It will stop itself.  Don’t hit stop on a take that you want to keep.  If you hit “stop,” your video stops at that moment and you will disappear from the project at whatever point you hit “stop.”  This is a video, please keep in mind that you are always being recorded. Keep a pleasant, interested expression at all times and stay engaged throughout. Think of your audience as being on the other side of the camera. Send them love while you sing.
• When you select “continue” it will render for a time.  Then it will bring up a screen where it will play back everything you just did. You can choose to listen to it or not.  At this screen, you can decide whether you “really” wants to keep this take.  If you decide that you want to keep it (whether or not you have listened to it) you should just select the green check mark.  If you decide that you want to do it over again, you should select the red “x” and it will take you back to the previous recording screen and the work that you just did will be lost. You cannot punch-in and replace a portion – this recording must be done in one continuous “take.”
• If you select the green check, it will take you back to the first screen that you saw after importing the project.  This would be the moment when you could choose to record in another box if you want to keep what you just did *and* record another take.  If you want to record in another square, just click on that square and it will take you to the recording screen and your video will show up in the square you selected. NOTE: I allowed my singers to record multiple videos to fill out the recording. You may or may not want to allow or include this step.
• I would request that at this point, you go back to the little “speaker” icon and turn off the reference recording and any other participants by clicking on each square and turning the volume all the way off and listen to your recording by itself so that you can see if the headphones have bled back into the microphone at all. You can also listen for persistent background noise and double check that the recording level is not too loud.  I will also do this when you share it back to me, but it will save both of us some time if you check it first.
• Once you have finished recording and have selected the video you want to keep, it is time to “save and share.”
• Click on the “save and share” link at the top of the screen.
• At the top of the screen, you can enter a “caption” that should contain enough of the song title to make it recognizable and your name.  Below this is the option to select a song – don’t bother with this step.  Below that are 3 options for sharing:  Public, Private, and Collaboration.  Please select “Collaboration.”  Lastly, select “SMS” (or “e-mail”) at the very bottom of the screen.  If the keyboard is covering up these options, just select “ok” in the top right hand corner and that should make the keyboard disappear. Note: People will tend to ignore this instruction and select “Public” or “Private” and then you have to go through the rigamarole of requesting and granting access – pain in the ass, but no matter what I did, it inevitably happened repeatedly.
• Once you hit “SMS” the phone will render for a period of time (depending on the length of the song) and then it will upload. Finally, it should pop up as a text (or e-mail) message.  Put in my cell phone number (or e-mail) and send it to me.  Then type in the name of the song so that I know which song it is.
• At any point, if things get messed up, you can always click on the link that I texted to you to import the file fresh.  It’s there in the cloud forever (unless I delete it)
• A couple of things you can do if you encounter errors or strange glitches: Close and re-open the program. Restart your phone. Don’t get too frustrated – if you can’t make something work (after you have read and followed these instructions) please call, text, or e-mail me. I’d rather spend a few minutes troubleshooting with you than see you get frustrated.
• If you decide to use the app for yourself, I’ll tell you for free that if you click on “save and share” and then select “private” and save it to your “camera roll” it will render, then upload, then save a copy to your camera roll.  This copy in your camera roll cannot be then opened and edited in the app, but it is super useful, because part of the process is that it will upload a copy of your project to your acapella profile in the cloud that you can then download, edit, and share any time. So every time anybody shares their contribution back to me, I immediately save it to my camera roll so that I have a copy of every stage of the project in my acapella cloud.  This is my standard method of “saving” a project that I am working on.
• After it finishes rendering and uploading, it will also give you an option to save the audio from the file.  Pro tip:  If you save the audio as apple multitrack lossless, you can use this audio for any video you are producing and adjust the audio in your better quality video / audio editing software.  This is what I do.  I don’t use the mixed sound from the video – I use the audio that I save as the last step of saving and sharing.  That’s why I asked that you don’t adjust any of the audio properties inside of the app (except for the volume) because I will do all of that in Adobe Premiere Pro.
• At this point, you are ready to click on the next link that I have sent you and start the process all over again – except that the app usually remembers most of your settings.

Since this is going to be a 360 video, you will want to record and collect some 360 footage that you will use for the background. Some things to consider: Much of the video will be covered up by your singers and orchestra, so as you assess a scene, consider what might make a good backdrop and where your musician videos will go. Will they replace a building? What makes the video interesting as a backdrop? Is it a busy street corner? Is it some beautiful nature? Do you want to create a time lapse – like a sunset or some interesting cloud movement? Do you want to use the audio at all, or not? I chose not to, but there are times when I regretted that choice, thinking it would have been cool at the beginning and end of the video to hear some of the wind in the trees or the sound of a stream or the noise of a busy street corner. Your choice.

It’s time to edit your work. You will want to assemble the files (probably on an external hard drive). I discovered that I ended up with a file that was more than 750 GB and was too large for my hard drive, so I had to do my editing on a fast SSD external USB-C hard drive. Keep in mind that when you render this video, the cache files will be enormous. My cache swelled to an additional 750 GB as my computer rendered the video – you can quickly run out of space and your render job will fail, and your computer might even crash. For organizational purposes, I put all of the files for each of the 15 movements into a separate folder. I saved each of the final Acapella App projects to my camera roll and then chose to save the audio as Apple Lossless Multitrack. The upside to this method is that you can adjust the audio properties for each person individually. The downside is that you will have absolutely no idea which audio track goes with which video track. This is why you will have your performers state their name at the beginning of their recording. If you don’t do this religiously, good luck figuring out who is who! The last stage of saving your audio will allow you to upload the audio to your Google Drive. I had already created folders for each movement, so I put all of the rendered videos into the appropriate folders and named each file so that I would know which audio file project went with which video file. Then I went into my camera roll and uploaded each of the videos to my Google Drive. Supposedly, with a pro account, you should be able to upload the video in 1080 resolution, but I was never able to do this. As soon as I would upload the video to the cloud, that option was gone. So all of my videos were in 720 – but this was good enough, IMHO. If you figure out how to get around this, let me know! Another hint: As each project was completed, I immediately began uploading them to my Google Drive. This way, I didn’t have to go hunting for them and save them all at the end of the project – it spread the work out. Again, you will find that having the fastest phone and fastest internet connection will be a huge benefit, because you will spend a lot of time waiting for your device to render and upload. The iPad Pro is probably a good choice.

You will begin by creating a new sequence. You will want to choose the preset that matches your 360 video footage. For me, I chose the “VR” folder and then selected “monoscopic 29.97” and then “3840 x 1920 – ambisonics.” The ambisonics part is very important. Next you will begin importing all of your assets. This includes your 360 “backdrop” video(s) as well as all of the individual video and audio files for the track you are assembling, including the conducting video that you probably created months earlier. To prepare the individual Acapella App videos for importing, I opened each one in Adobe Premiere and used single performer squares from other videos that I trimmed to cover up my conducting reference video. Where I did not have available performer videos to do this, I used various graphics to cover up unused squares, rectangles, and reference videos. Whenever possible, I repositioned videos by adding multiple copies of the same video and trimming squares and resizing and moving them around so that I could trim off an entire portion of the video. My goal was to capitalize on every piece of available “real estate.” Each person has their own work flow – I did each movement as a separate sequence and then combined the sequences to render out the final video. The first thing I did was add the 360 reference video as the base layer in the sequence. The program might prompt you to change your settings when you move your background into the sequence, but don’t do that – if you matched your reference video’s resolution correctly, then you should be good to go. Then you will start adding in your other videos as new layers. You may want to listen to them as you add them to confirm that they are lining up. On rare occasions, I would have a video that had to be nudged a frame or two to get it to line up perfectly with the other videos – who knows why – but nearly all of the videos lined up perfectly on the first go. Once all of the videos are lined up, then I delete all of the audio tracks that came with them – because I used the apple lossless multitrack audio instead of the attached mixed audio. Then I counted up all of the audio tracks that I would be using for my project and went up to the sequence menu and added all of the tracks as “ambisonic” tracks. Then I went into my assets and selected all of my audio files and right clicked on them in groups and selected “modify” and then “audio channels” and changed the “clip channel format” to “adaptive” and put a check next to channels 1 and 4. Quick tip: Assigning the audio to channels 1 and 4 will put the audio directly in front of the viewer. Later I will explain how you can pan the audio to the location where the performer *actually* is placed within your 360 canvas. For the best result, you will want to be working with mono files – stereo files will work too – you *can* put the left channel to channel 1 and the right channel to channel 4, but it could create some unpredictable results. But if you are selecting multiple audio files for this step and some of them are mono and some of them are stereo, you will probably have to go back later and reassign them (for example, if your audio meter shows some of them only registering in channel 1 or channel 4.) When you are finished, select “ok.” Then you can begin dragging your “adaptive audio” tracks into your sequence. You will want to zoom in to make sure that you don’t accidentally drop them into a place that is a frame or two off from your other tracks. Then I go through and figure out who is on each track by listening to the beginning of the track for the performer’s name. Then I name each audio track (they must be unique names – it won’t allow you to duplicate any names.) Then I selected each track individually and used the keyboard shortcut, “g” for “gain” and selected “normalize all peaks to” and then gave it a decible level. For my project, I put in the timpani at “0”, the singers and concert master and trumpets at “-4”, the principle players at “-8” and the remaining players at “-12”. This gets you close, but you will probably want to do some subtle adjusting. For example, if any of your singers or instrumentalists submit audio that is too loud (clipped) it will sound too loud and they will stand out – even if they are registering at the same decibel level. Once you have applied all of the audio properties and effects, you will probably want to trim the audio so that you only use the audio when they are actually performing to avoid excessive background noise, page turning, stand and microphone bumping, etc. But you will want to leave 1 second of grace at the front and end for a nice “fade.” At this point, you will want to listen to the audio, because later in the project, you will probably find that there is too much going on to render it in real time. When you are satisfied, select all of your audio and select <shift> <d> to apply the default fade in and out to the beginning and ending of each clip. This will make the entrances sound a bit more graceful and make it harder to detect the audio tracks coming and going. Next you will want to arrange all of your individual videos onto the 360 “canvas” video.

To arrange your individual videos, go to your “effects” tab and open the “Immersive Video” folder. Drop the “VR Plane to Sphere” and the “VR Projection” effects onto each of your videos that are *not* already 360 videos. “Plane to Sphere” will automatically warp your video so that it looks normal in the 360 environment. “Projection” will allow you to resize and move your videos around within the environment. Please note that you *can* drag your videos around, but this will ruin the “Plane to Sphere” effect. Instead, go to “Effects Controls” and select the video that you want to place. Use the “Pan” “Tilt” and “Roll” to move your video around. You can also use the same controls under “Rotate Source” with the “Plane to Sphere” effect – but you will probably find that the pan, tilt and roll with the “Projection” effect are simpler to use for most of your purposes. To test what it will look like to the viewer, you will want to hit the + button below your viewer and add the “Toggle VR Video Display” button to your controls. This button allows you to see what the viewer will see instead of the warped master video. Hitting the button again will take you out of that view. Important note: My husband did this part of the video editing and he is currently in bed sleeping. Sometime soon, I will twist his arm and he will contribute to this article and tell you how to create feathered masks around your videos and change their transparency. Keep checking back. I thought that his work was super artistic and cool and I was happy to hand this off to him so that I could step away from my computer and rest my eyes and stretch my limbs.

Once you have your audio edited the way you like it and your videos placed, it is time to create the ambisonic adaptive audio magic. Delete any audio channels that you don’t need or are not going to use. You may want to play your sequence one last time and double check that all of your channels are showing up on (and only on) channels 1 and 4. If any tracks are not showing up on both channels (and only both channels) then right click on that clip and select “audio channels.” Occasionally, I saw a channel that was labeled as stereo, but only had audio in one channel or the other. In this case, check boxes 1 and 4 on the channel that is working. But in all cases, there should be a check box only in channels 1 & 4. Then go to your sequence menu and choose the “add tracks” option and add as many submix channels as you have regular audio channels. (If you want to take a shortcut, you could also create a submix for each of your Acapella videos and assign all of those channels to a single submix, but I’m not sure that the end result will be as good.) Then go down your list of channels. Each of your channels should currently be assigned to the “Master” track. You will want to assign each of the audio channels to its own submix (or as I said, a submix for each Acapella video – cheater…) Then you will go to each submix that you create and assign it to the Master. Then you will want to click on the tiny “>” mark in the upper left hand corner of your audio mixer window. This should create an fx drop down. At the very top right hand corner of each of your submixes, click on the drop-down arrow and select “special” and then select “Panner-Ambisonics.” Then go to your video preview window and choose the “Toggle VR Video Display” button that you added to your controls earlier. Drag the view until you find the vocalist or instrumentalist (or if you are cheating, the entire acapella video) that is assigned to the submix that you are currently working on. Imagine a cross-hair going through the video preview window and centering on the video from which the sound should be emanating. Take note of the number at the bottom center and right center of the video. Then go back to your submix and adjust the “pan” until it gets as close as possible to the number in the bottom center. Then, if you *really* want to be picky, choose the drop-down menu that says “pan” and select “tilt” and adjust the knob until it matches the number to the right of the preview screen. Take care to note whether the number is a positive number or a negative number and match it. Lastly, access the drop down menu one more time and select “roll” and adjust it all the way to 180 degrees (positive or negative – it doesn’t matter which.) I learned this through trial and error – my placements were always off and on a whim, I tried rolling the audio 180 degrees, and poof, the audio was exactly where I wanted it. Confession time: I truly cannot tell the difference whether audio is panned up or down – so the tilt adjustment step a) might not be necessary and b) might end up reversed because of rolling the audio 180 degrees. Since I truly can’t tell if a sound is “up” or “down” I can’t figure out a way to test it and see if it is reversed. I was just glad to get the effect I was shooting for – that sound is coming from the direction that it *should* appear in the video. If you figure out a way to test this and come up with an answer for me – please let me know! Pro explanation: When you assign a track to channels 1 & 4, you are positioning that track straight in front of the viewer. Then when you send that signal to the submix and pan the submix, the effect makes the track sound like it is coming from the direction to which you panned it. If you assign all of the singers or instrumentalists that are in a single video to a single submix and pan that submix to the central location of their video, it *should* work, but I suspect that it will remove some of the clarity and individuality and multitrack nature of your final project, but it will also reduce your workload. I never tested this idea, but let me tell you, I was sorely tempted – but I badly wanted the end result to be good, so I panned, tilted, and rolled each track individually. I never found a way to enter the pan and tilt numbers using the keyboard – and the knob was never as precise as I would have liked. If you figure out another way to do this, please let me know! I can tell you for free that “yes” I did try to add the “Panner Ambisonics” effect to the actual track rather than the submix – but this definitely did NOT work. It seems like a pretty awkward workaround, but the only other idea that I had was to edit each of the audio files in Adobe Audition and convert them to ambisonic audio on tracks 1 & 4 and then import them into Premiere as front-directional files, enable their output to all 4 channels and then apply the effect. I don’t know if this would work, but I’ll bet it would.

Now the finishing touches. Go to the Master track and select the top down arrow and choose “Reverb” and then “Studio Reverb.” There are 9 options for this effect and I have no idea what most of them do. My solution was to drop this affect onto a regular track, go into the effects window and choose the style I wanted from the drop-down – for my video I used “Great Hall” and then I wrote down each of the parameters and then manually entered them into the drop various drop-down menu items in my master track. In case you want to know, for “Great Hall” they are a) low frequency – 300; b) high frequency – 4,000; c) width – 85; d) diffusion – 70; e) damping – 100; f) decay – 6,000; g) early reflections – 20; h) dry output – 60; i) wet output – 40. You *could* assign reverb to each of the channels individually, but then each singer shows up in their own little room’s reverb. When you add it to the master track, you are basically putting all of your singers into a single room – at least that’s how I see it and justify it! Next, I disabled all of the video tracks and played back some audio to check the levels. I try to get it so that all 4 of the master tracks are running at a couple of decibels below the maximum, and VERY occasionally peaking past 0. In order to achieve this, I added a second drop down to my master track – “Amplitude and Compression” and then “channel volume.” A new drop down menu shows up and I adjust each of the 4 channels equally up or down. (I didn’t want to mess with changing the amplitude of the individual channels, because this could make the sound appear to come from the wrong place – so if you are going to change them, change all 4 channels equally.). For most of the movements, I had to decrease each of the 4 channels by about 4 decibels. The solo movements usually needed a little boost. Lastly, there is nothing worse than digital distortion, so I selected a 3rd effect for my master channel – also under “Amplitude and Compression” you will find “hard limiter” and I left all of the settings at their default level. You don’t want this to have to engage frequently, but this was my insurance policy against any digital distortion.

If your computer can handle it, try to listen to the whole sequence one last time to make sure that you like what you are hearing. I have a VERY fast computer, but I still had to disable all of the video – and even then, there was some stuttering going on here and there – but I was able to get the general idea of what the sound would be – although you won’t get a true idea until after you have rendered your video and uploaded it to YouTube or Facebook and listened to it on a device that supports ambisonic audio – like an iPhone or iPad using a quality pair of headphones.

Now it is time to render your video. If you are only doing one sequence, you can skip this next step, but if you are combining a bunch of sequences like I did, I created a new file, created a new sequence with the same settings. Then I went back to this file and created a new bin in my assets, selected all of my assets and dropped them into this folder, gave the bin a recognizable title, copied it, and then pasted it into the master project’s asset list – eventually I had a bin for each of the movements for Messiah. Then in your master project, look at the top left hand corner of your sequence for this icon: (see the icon that looks like a stack of bricks with a slash through it?)

Click on that icon so that it looks like this:

Then, one at a time, go into each bin in your assets and find the 360 timeline that you created for that movement and drag it (in order) onto your master sequence timeline. When I had all 15 movements of Messiah (plus a very boring curtain speech) added to my timeline with no gaps between them (you may want to zoom all the way in to make sure that there are no gaps hiding in there.) Then I selected everything in the sequence and hit <shift> <d> to apply nice fades from sequence to sequence. Helpful Hint: The reason I did each movement as a separate sequence is because there were different performers in different locations for each sequence. I could not have put this all into a single sequence because I have to pan the entire audio channel to each physical location – if the singer or a musician shifts to a new location, I would need to move their audio to another track, assign it to another submix, and then pan that submix to their new location. Since nobody moved within a single movement, I could just set their location and be done with it and start with a new plan for the next movement – when the singers, instrumentalists, and backdrop would all have new locations.

If you only have one sequence, now is the time when you can rejoin this discussion. When you are ready to render your media, select your sequence or your master sequence, and hit <command> <m> on a Mac or <control> <m> on a Windows machine. A new window will open. Under format, select “H.264” from the drop down. Under preset, select “VR Monoscopic Match Source Ambisonics” from the drop down. (Unless, of course, your 360 video uses a stereoscopic style – then you would select that.) Under “Output name” give your file a name and location. As I mentioned earlier, I had to save everything (including the rendered file) onto an external drive, because the files and cache files were enormous. I chose “Render at maximum depth” “VBR, 2 pass” and “Use Maximum Render Quality” but left everything else alone. Then I crossed my fingers and selected “Queue” and then, when I was given the option, hit the button to start the rendering. Then (at 1 AM) I went to sleep and waited. And waited. I bought a 2020 iMac for this project. It has a 3.6GHz 10-core 10th-generation Intel Core i9 processor, Turbo Boost up to 5.0GHz
128GB 2666MHz DDR4 memory, and Radeon Pro 5700 XT with 16GB of GDDR6 memory. It took this computer 24 hours to render this video. When I uploaded it to YouTube, it took another 24 hours over a 6 Mbps (upload) connection. Then it took YouTube several days to process. The first time I uploaded it, I used YouTube’s Premiere feature and I had it uploaded a full 3 days before the Premiere. When the time for the Premiere came, the video still had not finished processing on YouTube and the Premiere failed. YouTube gave me a message that the video could not be processed and the only option was to delete the video. I had 200 people online and waiting for the Premiere and they were all commenting and wondering why the video didn’t start. My worst nightmare. I had to send a message and ask them to keep checking back. YouTube doesn’t have any kind of support team that you can call for help. It was truly heart-breaking after my team and I had put in thousands of dollars and months of work. I then began uploading the exact same file a second time – but this time as a normal public video (not a Premiere.) 24 hours later, the upload finished, and a few days later, our video finished processing and went public just like any normal video. Lesson learned.

By the time your video goes public on YouTube, your publicity teams will want to have articles in your local newspapers, sent press releases to your local TV stations, sent out blast e-mails. You will want a big buzz on social media creating anticipation. Your members will want the link to share via e-mail and text messages. You will want to tell people how to watch a 360 video with ambisonic audio. Maybe your husband will create a cool “how to” video like this one:

Maybe you will send out care packages with 360 viewers (like Google Cardboard) and party favors to your top donors. You will want watch parties and so on. Anything you can think of to get out the word. Have a raffle, have your singers upload videos of themselves singing in the shower. Send the link to your sponsors and let them share it with their clientele. Facebook is pretty unfriendly, so you have to trick them into helping you. One of our orchestra members used her phone to take a video of a clip from the 360 video and she shared that clip on Facebook and it did really well. (Go Heidi!) I promise you that if you just share the URL of your video on Facebook, NOBODY will see it. You *might* get some views from Facebook if you have a group page and you can do a paid promotion. And if you are *really* desperate, you might spend an entire day creating a “how to” page on your personal website in order to garner a few more ongoing views – LOL! In all seriousness, if you have gotten to this point in my blog post, you are one dedicated human being. Congratulations! If you have questions, please comment on this post and I will do my best. I have never taken a course on how to use Adobe Premiere and I learned pretty much everything I know about 360 video and ambisonic audio from YouTube videos and blog posts like this one – except when I did this video, nobody had ever tried this before, so some of it I just had to figure out on my own. It was a fun project that took 3+ months of my life during the crazy COVID-19 pandemic and gave me an artistic outlet and something to make me beat my head against the wall. But I’ll tell you something else for free: As the director of Aspen Choral Society, I am normally leading a rehearsal in front of 75+ people and I don’t even know their names or much about them. For this project, I dealt with many of them one-on-one and got to hear their individual voices, and all of those individual connections mean the world to me now. When I put on my tux and went out into nature to record myself conducting the movements, people probably thought I was some (relatively safe) kind of crazy. (They’re not wrong.) But they would come up to me and talk to me and ask me what I was doing. I got to make more connections that way. If I had been smart, I would have brought somebody along to video this happening and they would have uploaded the video to our Facebook page. I hope you will learn from my missed opportunities and create something even more amazing and wonderful. If you see this blog and use my ideas, take a second to like our video and comment on it – that can be your “thank you” back to us – because in the end, that’s why we musicians do what we do. Who are we without our audience? Let your light shine!