EP003: Multi-Site Workflow

Episode 003
Duration 43:23

An overview of how we manage a multi-site Resolve-focused workflow

Over the past few years, like many of you, we’ve worked a lot at home.  At first, this was by necessity (pandemic) but then became more of a lifestyle choice. In the greater Washington D.C. area where we are based, it’s not uncommon for people to spend 2-3 hours in a car, stuck in traffic commuting to and from work.  Yuck!

Years ago our workflows were based around a single location – our studio in Silver Spring, MD.  This meant not only was that where we worked day in and day out, but it also meant that was where all data lived and flowed in/out of.

That centralized location was great when everyone on the team was coming in every day, but when the pandemic hit we quickly had to pivot to figure out how to best work from home. That’s progressed to now we are pretty much location agnostic.

We get asked all the time about the components that go into a multi-site workflow like this.  In this installment of The Offset Podcast, we discuss the main pillars of our workflow.

This includes:

•Leveraging the Blackmagic Cloud to host our project databases
•NAS setups leveraging TrueNas
•Using SyncThing to keep things between locations in sync
•The importance of fast Internet

There are some other areas that we didn’t cover in this episode but will for future episodes including real-time streaming,  leveraging a switcher to control a remote session, and more.

We hope you enjoy listening or watching.  If you like this episode please like and subscribe if you’re using a platform like YouTube and tell your friends

-Robbie & Joey

In our next episode…

You’ve probably been living under a rock if you have on some level been engaged in conversations around machine learning and artificial intelligence (we have strong feelings about that last term!).  In our next episode, we take a look at ML/AI our feelings on it, how it is currently and will impact postproduction and why the hysteria about ML/AI is not always warranted, but sometimes is.

Video
Links
Transcript

01:00:00:07 - 01:00:16:01
Robbie
In today's episode, we're talking workflow and not just a generic workflow, but the one that Joey and I actually use every single day on our projects. Stay tuned.

01:00:16:02 - 01:00:17:20
Robbie
Hey there. I'm Robbie Carman

01:00:17:22 - 01:00:18:18
Joey
And I'm Joey D’Anna

01:00:18:21 - 01:00:45:12
Robbie
And welcome back to another episode of the Offset podcast. Joey, today we are talking workflow. This is something that people bandy about. I mean, every single day in our circles and post-production and, and finishing. And there's a lot to workflows, obviously, but we get asked, I mean, I think pretty much weekly about the workflow that we do between ourselves and our different locations.

01:00:45:14 - 01:01:08:11
Robbie
And I want to kind of explore that today because we answer a lot of questions, like I said about it, and we're not doing anything as far as we're concerned, super special. We're just kind of linking together a lot of, you know, sort of disparate pieces to kind of make things more efficient. But before we go into our part of the workflow, let's talk about what the kind of situation that we're facing is and some of the challenges.

01:01:08:13 - 01:01:26:18
Robbie
You know, that we're we're trying to conquer because there's there's a few of them. So so everybody knows we have an office. You know, it's kind of our main office, you know, facility where we do a lot of our reviews. We work from there. That's where our whole audio team is based. But as you can see, Joey's room right there is in the basement of his house.

01:01:26:20 - 01:01:52:15
Robbie
I have a similar set up in the basement of my house. And so we have these three locations, but we're constantly meeting to have media in one place and media and projects and another place. And, you know, for years, Joe, I think I don't want to say we struggled because we had some we had some workarounds. But why don't you describe kind of the kind of old way of doing things and how we got that and then we'll jump into what we're doing these days?

01:01:52:17 - 01:02:17:13
Joey
Yeah. So like Robby said, we've been kind of doing a multi-site shared workflow for a very long time. Both of us started working from home before the pandemic, and we've been moving media around, moving projects around and working together on projects in various different ways over the years. You know, early on it would be as simple as, okay, here's all the client media, right?

01:02:17:14 - 01:02:20:14
Joey
We're going to download it. I'll download it on my end, or you might.

01:02:20:17 - 01:02:21:14
Robbie
Drive.

01:02:21:16 - 01:02:38:20
Joey
Or drive or drive and I'll copy the drive and then move the drive over to the office or give it to Robby so he can take it to his house. And I would maybe set up a project on my end, organize all the folders, how I felt like organizing them on my storage. Right. And then Robby would do the same thing on his end.

01:02:38:20 - 01:02:57:18
Joey
And when we needed to move something back or forth, we would export a DRP, a resolve project, and then he would open it on his end. Re link all the media. Sometimes that was an easy process, sometimes it was not, depending on if we had used different naming or different drive lettering or, you know, going from different platform to platform.

01:02:57:20 - 01:03:19:09
Joey
And we would do that back and forth. So when it would have to come back to me for any reason, I would have to do the inverse and basically undo all that Robby did to make it work on his end, to make it work on my end. And there was a lot of inefficiency involved there. So we've gone through both as the software as evolved and as our processes evolved internally.

01:03:19:11 - 01:03:24:19
Joey
A couple of really important steps to make all of this basically seamless.

01:03:24:21 - 01:03:41:08
Robbie
Yeah, And I mean, I think there's a couple of things that, you know, early on, you know, we were both of us kind of, you know, we've been working, you know, together for almost ten years now. And I think, you know, there was things that you like to do in the way that you name things and organize things or things that I like to do.

01:03:41:10 - 01:04:01:18
Robbie
And, you know, you know, I think we got our shorthand down pretty well. But there were still sometimes, you know, some detective work. Okay. Like where did you put these, you know, these patches that came from the client or you got audio mix from the audio team. Where where does that go? You know, and so there were some organizational things besides just the mechanical stuff of like how we how we push stuff around.

01:04:01:18 - 01:04:20:11
Robbie
And I think, you know, my biggest pain point from that period of time was especially with resolve and drops and projects, Files is staying in sync, right? Knowing kind of this is the latest version of this project. This is the latest timeline. that's the one that you did X, Y and Z on. And that was a real challenge.

01:04:20:11 - 01:04:36:04
Robbie
And we'd end up at the time, end up with a folder of, you know, 20 or 30 drops from after reuse review session, post review session, and we call it different things. And it got super confusing real quick and we said, No, no, no, we got to. Yeah, and that.

01:04:36:04 - 01:04:54:23
Joey
Kind of collaboration kind of discouraged the teamwork aspect a little bit, right? Because if it's a process to go from one site to another and back, I can't say, Hey, Robby, can you look at this shot on this timeline real quick? You know, it might take me less time just to figure out my problem myself or vice versa.

01:04:55:04 - 01:05:18:09
Joey
These days, we share very small components of projects really regularly. can you work on this masking problem while I finish coloring this act of the show? Can you do Act one and I'll do Act two? You know, basically in our day, if we need help, we can always reach out to the other one for help. And because the workflow is seamless, there's no inconvenience penalty to doing that.

01:05:18:09 - 01:05:32:11
Joey
Whereas with the old DRP way, it's kind of like the problem has to be big enough to justify the back and forth before we collaborate, whereas now it can just be like, Great, you do this, I do this, we get it all done quick.

01:05:32:13 - 01:06:01:04
Robbie
Yeah, and to be clear, I think a lot of people might be listening going, Well, yeah, I do that every day in my facility. That's it's easier to do when you're down the hallway from somebody and you're working off the same shared storage and all that kind of stuff. Harder to do when you have two people in different locations and in our case, actually have multiple locations that we might not even be at every day that we want the media, they're like, I want to be able to go to the office, you know, and not have to move anything over or do anything and just sit down at the desk at the office and go,

01:06:01:06 - 01:06:20:19
Robbie
yeah, those were the projects where I left it last night, or we left it last night at home and we're off to the races. So I think for me, Joey, kind of we can break this down into a number of parts. And I think the hardest one for a lot of people to kind of get their their head around is the idea of sinking media, right?

01:06:20:19 - 01:06:40:09
Robbie
So, of course, for the longest time, everybody knows the phrase seeker net that was the way it was done, Right. Here's a you know, a thumb drive or hard drive. You move it from one system to another system. There's lots of problems with that approach. So I think you hit on one of them and that is simply different platforms, different drive matching mapping.

01:06:40:11 - 01:06:42:23
Robbie
Can you expand on that a little bit?

01:06:43:01 - 01:07:02:16
Joey
Yeah. So one of the things we haven't mentioned yet, though, I think is important to kind of add into this overall equation is not only do we have three different sites, we have two. And it's sometimes depending on how we've configured machines, three different platforms. Yeah, we might have a Linux resolve, which we've had on and off in the past just for various reasons.

01:07:02:20 - 01:07:35:19
Joey
But right now, primarily we're on Mac resolves and Windows resolves In my suite here I've got one Windows Resolve and one Mac resolve. At the office we have just a Windows resolve at Robby Suite. He's got two Mac resolves and they all address media differently. So the big problem with storage is multifaceted. One getting all of the information for a project, basically all of the files to all three sites and when one changes somewhere making sure that change happens everywhere.

01:07:35:19 - 01:07:57:06
Joey
So if I download mixes, I need those mixes to be sent to the office and Arabes house. I need Robby to be able to download a patch from the client and then I get it and the office gets it without us having to do any kind of manual intervention. That's one part. The next part is file paths and drive mapping, and that kind of takes two things.

01:07:57:08 - 01:08:22:05
Joey
One is a technical thing and one's an organizational thing. The technical issue is, Max, some pieces do drives differently. You could have like the D drive or the E drive on a Windows machine and then on a mac or Linux, it's slash volume, slash, whatever, slash M.A., slash, whatever. However you mount the system. Now, Resolve has some really great features for dealing with this called mapped mounts.

01:08:22:07 - 01:08:44:16
Joey
And you can basically define all your storages and also what they are on other systems. So for example, on our systems, on the Mac and Linux is it slash volumes slash DC color, Right. And if we get on to a Windows system, we have the map mount set to that and on the max we have the math mount set to our drive.

01:08:44:16 - 01:09:14:10
Joey
The reason why we have our drive is because all of our Windows machines have the DC color stored folder, as are. That way we're only transposing paths between Unix style paths and window style paths on Mac and Linux. It's always in the slash volumes folder on Windows, it's always on the R drive, so we only have to do one mapped mount and that's to associate between different platforms, not setting up map mounts between my house and the office.

01:09:14:10 - 01:09:16:19
Joey
The office and Robbie's house. My house and Robbie's house.

01:09:16:21 - 01:09:40:22
Robbie
Totally. I think there's a few things to kind of further explore their Number one, we do not operate with working off of external removable drives, right? I think that's a challenge that a lot of people find themselves saying, like, well, I had this drive that I'm plugged in. Like this whole idea that we're about to expand a little bit more detail is predicated on the idea that we essentially have big storage.

01:09:40:22 - 01:10:08:20
Robbie
We have a the facility, we have a mass at your house and that's at my house. And so those file paths once set up are kind of always the same. Right? They don't they don't change. Right. You know, slash volume, size, DC color is always there slash our our our colon slash flash is always there. I'm on the windows boxes and then too, because I think some people might be new to mapped mounts and resolve but I'm always I mean to this day I'm surprised how many people are like, whoa, I had no idea you could click on that little final thing and it changes.

01:10:08:20 - 01:10:15:11
Joey
It's been in there since like version six. And it's I don't understand why every other platform in the world.

01:10:15:11 - 01:10:17:06
Robbie
Doesn't do that, doesn't do that. So it's so.

01:10:17:06 - 01:10:19:12
Joey
Simple, but it's such a timesaver.

01:10:19:12 - 01:10:42:15
Robbie
It is just I mean, the simple way of saying it is that once you set up a map mount, if it's so for for example, if you're trying to map, you know, volume slash DC color to R and you set that are up as your math mount, basically the way it works is that any time in a file path it sees slash volume, slash DC color, it goes, I know, I'm going to swap that out with Star or vice versa.

01:10:42:15 - 01:10:59:20
Robbie
Right? And so the real advantage of that map out workflow part of this is that as we explain in a second, once we start sinking media, is that we never have to re link anything, right? As long as that mapped mount is set up, I just open up the project even if I'm on a different platform in those, Right.

01:10:59:22 - 01:11:22:14
Joey
Yeah. And that's where the standardization part of this comes in. You remember. Right. So there was two things. There's kind of the technical part, which is the mapped mount and then the operational part, which is standardization. One, we can use the mapped mounts because we standardize mount points across all of our platforms. Like I said, all the Linux, all the Windows, all the Macs have the same mount path within their ecosystem.

01:11:22:20 - 01:11:51:04
Joey
So we're just using mapped mounts to go between them. But the other thing that took us a long time to kind of really commit to, but I think it's really, really, really important when you work with a team, even if that team is just two people. Yep. Is we now have a dedicated templated out project folder structure. And what that means is every project gets the same folder structure and the first, you know, it's kind of like using a fixed node structure in resolve, right?

01:11:51:06 - 01:12:17:12
Joey
When you first think about it, you're like, Well, that's going to be really limiting, but it's not if you build it out oversize. So our our folder structure basically has a dedicated folder for anything we would ever need in a project, in any given project, we might use 10% of those folders. But one thing I've found over years of working in post-production is the best way to be organized is to automatically have a place to put things that is the right place.

01:12:17:14 - 01:12:43:00
Joey
If there's a right place to put things, that becomes your default. So even when you're under the gun, even when you're rushing, you just jump to the right place first instead of jumping to, God, I got to get this done quickly. I'm going to just throw it on my desktop or I'm going to do something disorganized. So because now the project folder structure is the same in any site that we have, the project, the maps are the same, the mounts are the same.

01:12:43:02 - 01:12:59:22
Joey
It really is. I can open up a project on my desk. I can walk over to my assist station desk, which is a Windows machine, open up the same project. I can drive to the office, open up the same project, and I can call Robby and say, Robby, can you look at shot number two on this project? And he can literally just open it up and go to shot number two.

01:13:00:01 - 01:13:02:21
Joey
And we're all on the same page with no rethinking of media.

01:13:02:22 - 01:13:27:13
Robbie
Totally. And I think that it's, you know, that how we have that organized. So we're using a tool called posthaste, that popular tool on the Mac platform people have known about for a long time. It's just it's allows you to automatically create those those folders structures however you want. That's a good point to make, is that we basically organize things for client provided stuff and then stuff that we create or organize on ourself.

01:13:27:13 - 01:13:48:12
Robbie
So in the client folder in my we have a folder for, you know, a consolidated media folder, if they give us like a or pro project or something like that, we have a folder for baked files, we have a folder for, you know, fixes and VFX and stuff like that. And in our, in our folder we have, you know, folders for audio and graphics that might be received from other vendors.

01:13:48:14 - 01:14:09:10
Robbie
We have a folder for screeners and final output. The point of this is not to kind of, you know, lock in, you know, you know, for people to take our folder structure, it's more so just to think about that folder structure that works for you and our unit on your end and then sticking to it, right, that if you know every time, wow, I'm looking for the reference file for this timeline.

01:14:09:14 - 01:14:18:07
Robbie
Well guess what? It's in client stuff and references and there you go. Right. And it's just that repeatable and is easy to find.

01:14:18:08 - 01:14:40:10
Joey
And it did take us a while to get to this point. And it was for no reason except that we were working right. So we were working good enough, projects were coming in, projects were getting done, projects were going out. There wasn't a dramatic issue to solve. So we never just sat down and said, okay, let's pause. Think about the actual workflow, the folder structure, and standardize.

01:14:40:10 - 01:15:04:00
Joey
Yeah, once we did that, we did a couple iterations, right? We tried some things, tried it with some real projects, evolved it and then kind of locked it in. And that's important. Any time you're kind of building a templated workflow, try it. Be ready to make changes. Eventually you're going to lock it in. But you know, we have some early projects that might not adhere to the structure exactly, because we were still wiggling around and playing with it.

01:15:04:00 - 01:15:11:10
Joey
And that's fine. It's always going to be a process, but just making the effort to have everyone on the same page makes a huge difference.

01:15:11:12 - 01:15:26:04
Robbie
And I think it's kind of like a diet or an exercise plan, right? Like I'm more guilty than you are of like sometimes just being and getting lazy and going, I'll throw it in. But, you know, it's one of those things. The more that you do it, the more it becomes habit and the more you see the benefits from it.

01:15:26:04 - 01:15:40:04
Robbie
Just like a diet and exercise plan or whatever, you know. And I think that, you know, that iteration is important. You know, we did early on, for example, like I didn't realize that like lowercase folder names drove you crazy. Right? And, you know.

01:15:40:04 - 01:15:45:02
Joey
It's like, which is completely irrational. But I just I like uppercase folder names. I don't know.

01:15:45:03 - 01:16:00:14
Robbie
All right. So we got you know, so we've we've covered the part about map mounts. And in the file past, we've talked about sort of a standardized folder structure. How are we actually getting media from point A to B to C? How does that actually work?

01:16:00:16 - 01:16:31:09
Joey
So everybody knows there's lots of solutions for this and the right answer for you is whichever one is right for you. But for us, our needs in general were we don't want to go to a cloud provider, we don't want to mirror everything in the cloud because that's either going to get really slow or really expensive. Yeah, we needed to be able to reliably go between three different places and we needed to be able to kind of control it because, you know, we have a lot of really huge projects that don't ever need to move.

01:16:31:09 - 01:16:52:04
Joey
Like I might be working on a film for a long time and never need to touch it at the office or Robbie's place. And we don't want to just sink the entire pile of everything we always work on. We want to be able to say, okay, this project checkbox goes to the office, goes to Joey's house, goes to Robbie's house, This project goes to Robbie's house in the office.

01:16:52:08 - 01:17:20:17
Joey
Right? So we need to be able to kind of direct what projects we're putting to what storage, because, like Robbie said, we have nasties at each location and while storage has gotten really cheap, it's not free and media files are doing nothing but getting bigger. Yeah, so we do need to be able to do some management first. We started using Resilience Inc back when it was still called BitTorrent Sync originally.

01:17:20:21 - 01:17:22:03
Robbie
That's right.

01:17:22:04 - 01:17:33:02
Joey
And Resilience Sync is a pretty cool product. A lot of people use it very successfully. We ended up switching off of that because we had some various technical problems with it and we had also.

01:17:33:04 - 01:17:51:01
Robbie
I should add that we were also running resilience sync locally on another NAT systems, but on a computer that was at that particular location. So there was a mac, a spare mac or PC, wherever there's various locations that was running resilience sync as an application.

01:17:51:03 - 01:18:13:01
Joey
Yeah. So to manage it, you'd have to go to that computer, right? So what we ended up settling on is a really cool piece of open source software called Sync Thing. And what Sync Thing does is basically it runs in the background on any system, which also means we can run it inside a virtual machine on our actual NAS systems, and this works on various NAS platforms.

01:18:13:01 - 01:18:34:22
Joey
So if you're on Synology Qnap, Troon apps, which is what we use, most of them have virtual machine capability and you can run sync thing right on your NAS, which means it's super fast indexing files, scanning files, sending files back and forth, and it doesn't put a load on your grading system. And what same thing does is it presents you with a web UI.

01:18:35:00 - 01:18:59:21
Joey
You can access the web UI from anywhere in your network. So I can like jump on my iPad and say, Hey, Robbie, I'm sending you this project click and it's done because all the paths are available minus and sync thing is running on the NAS and we can just basically have a we just have a running list of projects where they're saying to and if we make changes, it watches the folders, uploads the files to and from different locations.

01:18:59:21 - 01:19:19:07
Joey
It has a trash functionality that has saved us in the past because let's say somebody deletes something at a remote site, then it's going to ripple that delete to the other sites to keep everything in sync. Well, you can set a preference in there to have it instead of delete it, move it to a hidden trash folder. That way, if I delete half the project, guess what?

01:19:19:07 - 01:19:28:05
Joey
I might have deleted it from my system. It's still at the office. It's still at Robbie's house because it didn't delete them from his storage. It just moved it into the trash folder.

01:19:28:05 - 01:19:47:21
Robbie
That's a good point. I'll expand on that just one second. So I think it's important to make note of a phrase that's often used with this kind of technology and that distinguishes itself from other ways that people might be working. You know, this is a peer to peer kind of setup. We are going from minus to the office.

01:19:47:21 - 01:20:07:18
Robbie
Nice. Joey's nice. Wherever. Wherever this client's running directly, We are not using a relay point first, you know, so people might go, yeah, sure, I collaborate media, but I might first sync it to, you know, my Google Drive or Dropbox or frame my well, in that situation, you're going up to some point relay point in the cloud.

01:20:08:00 - 01:20:30:20
Robbie
And then the other end is having to pull that down. So you have three stops along that chain as opposed to and we have noticed that from a speed perspective, it's it's as fast as the pipes on either end can support, you know. And so for for us, that basically means gigabit speeds, you know, to and fro. And so, you know, we've.

01:20:30:20 - 01:20:44:23
Joey
Got seeing progress files move at 900 megabits a second on sync thing is is pretty cool because it's like Robby can you send me that project you want me to have a look at. Sure. Click. Yes. Okay, cool. I've got it. If it's a small project.

01:20:44:23 - 01:21:01:19
Robbie
Yeah. And I think so. I think the added benefit, as you mentioned, that of running this on your NASA's systems too, is that those machines, those boxes are generally going to be running all the time. So there's not a concern of, that computer went to sleep or somebody, you know, took that laptop off the network or whatever it is.

01:21:01:21 - 01:21:34:09
Robbie
These are kind of fixed assets. They're always listening and watching for media come in. And I think you mentioned the trash thing. The other thing I think is a really important thing that we've gotten in trouble with in the past with tools like resilience, was how these these these sync tools handle conflicts, right? So like, you know, if, for example, you're thinking something while it's still downloading or, you know, you created a folder at the exact same moment that I created a folder sync thing has a robust set of tools to kind of manage that too, you know, Hey, I noticed that this file is out of sync.

01:21:34:09 - 01:21:38:10
Robbie
You might need to look at this and address this, and that saved your butts a few times as well.

01:21:38:12 - 01:22:00:00
Joey
Yeah. And because it's on this nice web based dashboard, like a lot of times we can actively manage it really easily. Like if we are rendering a hour long process for, for, for file, I'll just click pause on that sync because I don't want it trying to scan that file while still being rendered. Right. And then have to figure it out at the end and re scan it again.

01:22:00:05 - 01:22:15:03
Joey
So it's just it's always there, it's always running, it's always in the background and it's very easily accessible. I can even VPN in and grab it from my cell phone because all we're doing is telling sync thing what to do. We're not moving data around on that actual machine we're accessing it from.

01:22:15:03 - 01:22:32:07
Robbie
Yeah, And so from a practical point of view, if let's just as Joe mentioned earlier, when we first started talking about the ability to ask for help or get a fix on something in that in that sense, what we usually do is, Hey Joe, I need you to whatever, do some magic, you know, some VFX magic on the shot.

01:22:32:09 - 01:22:48:17
Robbie
His reply. Well, okay, sure. Just sync the project over to me. I click two buttons and then in just a matter of moments, you know, again, depending on how big the project is, it's over to him and vice versa. Let's say I get a client who sends a drive to my house, but I'm going to be reviewing with them at the office.

01:22:48:19 - 01:23:16:02
Robbie
Well, that's easy. I'll just copy the drive here, hit sync to the office, and when I go to the office tomorrow morning, guess what? Everything. They're already ready to go. So it's very valuable in kind of moving that data around. And I think combined with the two other things you mentioned earlier, the final pass stuff and the fixed you know, the fixed folder structure, it's like working on, you know, really it's like working on the same system everywhere you sit down all the time.

01:23:16:08 - 01:23:20:00
Robbie
The media is just there, which is which is killer.

01:23:20:02 - 01:23:38:20
Joey
Yeah. And I think that's kind of a great lead in to the other big, big, big, massive ingredient for this that is relatively recent but is conceptually very, very important. And that is how do we interact with the projects across multiple sites.

01:23:38:20 - 01:24:03:01
Robbie
Yeah. And as you said earlier, you know, it used to be this mental gymnastics that you had to play of. You know, is this the right version of the project? What's the timeline in that sequence? Like we you know, I think at the time we were just like, God, this could be so much better. And, you know, we've, you know, covered I know we've done a lot of training about like the true collaborative workflow and setting that up and resolve.

01:24:03:07 - 01:24:09:04
Robbie
But that's all been predicated on the idea that you're working on the same network, right, That you're in the same place.

01:24:09:09 - 01:24:38:04
Joey
And we went down this rabbit hole for years trying to set your own VPN, cloud based shared database server, and we could never get it to perform at the level we needed it to because the Resolve database connection to PostgreSQL, which is an industry standard database, but the way Resolve talks to it is very important and that was never fast enough over high latency connections like the internet Resolve 17 I believe it wasn't.

01:24:38:04 - 01:24:39:09
Robbie
17, I think it was just.

01:24:39:11 - 01:24:55:15
Joey
17 completely rewrote that connection specifically to make it fast enough to use over the internet. And that's what allows the black magic cloud to work. Yeah, but it also would allow you to make your own private cloud database server if you wanted.

01:24:55:17 - 01:25:13:11
Robbie
Yeah. And so we decided after looking at that for a long as you said, we had tried VPN, we tried all sorts of things. When the Black Magic Team came out with Black Magic Cloud and supporting databases there, you know, I think the first thing we thought of as well, we don't want to pay anybody extra money to do anything and well, yeah.

01:25:13:11 - 01:25:16:09
Joey
We could build this ourselves. You can build out now that it's possible, right?

01:25:16:14 - 01:25:22:17
Robbie
We would build this themselves. And where did, where did we land with that and why did we land on what we did?

01:25:22:19 - 01:25:47:06
Joey
I, I started sport up virtual machines, VPN servers and we were going to do it all in-house because again, the technology and resolve that allows the cloud to work will also work on a private database server. There's nothing different between the two, so you can do this privately. Once I got through all the kind of I.T. side and dealing with, okay, do I want all three offices on the same VPN?

01:25:47:07 - 01:26:11:03
Joey
Right. And then that all the time. Probably not. What if I want a VPN into my own house from my phone for other reasons? Do I want to have three different VPN servers running? Do I want to actively manage and secure all this? Yeah. No. If you have a big I.T staff and you know, kind of an already done cloud infrastructure for other things, that's a great option.

01:26:11:03 - 01:26:35:15
Joey
But for us $5 a month per database and you just log in. So we're on black magic cloud for essentially all of our projects. Yep. They're all hosted on the cloud database. We still do local backups of everything just to be safe. But I mean, for the money, it is just, yeah, it's not worth our time to become I.T. experts to do that.

01:26:35:18 - 01:26:41:08
Joey
And also, it's not worth the risk of exposing our networks to security problems.

01:26:41:10 - 01:27:11:06
Robbie
Yeah, and I think there are some people who, for whatever reason, they're in a TPN situation or they have to be air gapped for other security reasons. You know, there are valid reasons to recreate this kind of technology internally where it's under your control and, you know, you have a little more, you know, security. Throw at it with that said, you know, black magic is using, you know, kind of industry standard as kind of backbone for this kind of stuff, you know, So it is secure, easy to use.

01:27:11:06 - 01:27:20:06
Robbie
As you said, the cost $5 per database, which I think correct me if I'm wrong, but I think that supports what, at $5 you get ten users per database, something like that.

01:27:20:08 - 01:27:40:05
Joey
In shared users and infinite projects, right in the database in that database. From a practical perspective, we've been creating a new database basically like per major version of resolve and backing up the old one. But I mean, at any given point we have one or two databases, so five or $10 a month.

01:27:40:07 - 01:28:00:05
Robbie
So now in practicality, now all this really means is that inside of resolve, instead of opening up a local database or a shared database, you know, from a server that's on my network or on the network, come on, we're just pointing that logging into the black Magic cloud, it operates and functions just like any other database, you know, SQL database that you might have and resolve.

01:28:00:07 - 01:28:20:19
Robbie
But here's the benefit, right? So I go ahead and I sync a project over to Joey and let's call it, for lack of a better term, project X, Y, Z. I put it in that database. I add some media to it again, that's on my maps. So first thing I do is I'm clicking in sync thing to share that folder from minus over to Joey's again, standard folder in a few minutes, Joey gets that folder.

01:28:21:01 - 01:28:40:01
Robbie
All he has to do on his end is open up resolve, find the project that I just created. He opens it up and guess what? Through the beauty of path mapping and all that other stuff, everything's online. He simply slash thing work. He simply smacks me or texts me and says, Hey, Rob, what shot was it? I go, He was 17.

01:28:40:03 - 01:29:04:01
Robbie
He goes, Cool, I've done a pass on that. Open it up. See what you think. In fact, it works so well for doing this that I can open the project in read only mode. If he's happy he has an open and just look at what he's done. But it gets even better than that because the cloud database of course, also supports true collaboration mode, which prior to this was really kind of limited to.

01:29:04:01 - 01:29:28:23
Robbie
The development that you spoke about was really limited to doing collaboration on a single network, right? You have a shared database server on the network. Everybody in that facility or places could do that. But now we can replicate all of those true collaboration features through our cloud database. So if we want to work at the same time, for example, I'm like, Joey, can you follow behind me, you know, doing this roto work and fusion because I don't know fusion at all.

01:29:29:04 - 01:29:38:03
Robbie
You can go, Sure. And all the features that we're used to in collaboration, how timeline ownership, how flip ownership works, all that stuff works like a charm.

01:29:38:05 - 01:30:08:05
Joey
Yeah. And performance wise, I cannot emphasize this enough. I can't tell that I'm working on a cloud database versus a local database. It's just they've figured out the caching and the dataflow so well that it just works seamlessly in single user or in collaboration. One last thing before we kind of talk about collaborating in some of the gotchas involved is I do want to mention you talked about TPN and air gapping and stuff like that.

01:30:08:07 - 01:30:31:17
Joey
There is a happy middle ground to be had with that because while we're not doing specific like major Studio TPN certified things, we still very much care about our client's data security and privacy and making sure that we don't get anything out in the world that it doesn't belong. And, you know, things don't happen security wise that are bad.

01:30:31:17 - 01:30:51:03
Joey
So as a general policy, we don't have our resolve systems connected to the Internet, right? My main resolve system is not online. I bring everything in and out and do all my email and everything else on a mac studio that's sitting next to it. But that brings up the question, well, if the main resolve isn't online, how can you do cloud database?

01:30:51:09 - 01:31:20:06
Joey
Well, the main resolve isn't online, but it's on the network. And I've actually created firewall rules and say, okay, do not let any Internet traffic in or out of this Mac Pro on this Ethernet port right. But allow connections to and from the Blackmagic Cloud database and that was like a five minute configuration in our firewall. You could probably do that in any, you know, firewall, maybe even on like your home Internet router.

01:31:20:06 - 01:31:55:03
Joey
It's not a very complicated thing of combination of ports and hosts to add kind of to the white list to say, hey, allow this, don't allow everything else. So yeah, if I open up a web browser on my main machine, can't see anything. If I try to do Mac OS updates, can't see anything. If I try to download a file, can't see anything, I open up, resolve it just pops up with a cloud log and I put in my username and password and I'm in so you can get to a reasonable middle ground of security, convenience and mostly air gapping your machine without being like, I'm going to put this thing in a prison and

01:31:55:03 - 01:31:57:12
Joey
only bring things in and out with certified thumb drives.

01:31:57:15 - 01:32:28:04
Robbie
Yeah, that's a great point. And one more thing that I just thought of before we get to the collaboration gotchas real quick is, you know, I think working in the cloud instills some fear in people in terms of like, well, what if, you know, black magic, creative use goes down or you know, what can I do? You know, the normal suggestions about project backup, whether that's, you know, obviously a live thing or an actual project backup or however you actually want to do it by exporting drops.

01:32:28:06 - 01:32:48:09
Robbie
All of those practices are still germane, of course, to working in the cloud. Just because you're in the cloud. I wouldn't depend on that. As you know, your soul, your soul reliable backup system. So one thing I don't think a lot of people realize that is possible because in, you know, in Paris you have to take a whole database and then you'd have to export that and makes that PTAB zap thing.

01:32:48:09 - 01:33:08:12
Robbie
It's big, it takes a while or you'd have to export individual drops. There's actually an easier way to do this that plays into the drag and drop nature. A lot of us is you can simply just highlight a project or even a folder or projects or even everything in the project manager window and resolve now, and you can simply just drag that to a location of your choosing.

01:33:08:14 - 01:33:23:15
Robbie
So, you know, if you want to just the first thing you do in the morning when you open up resolve, just take something, drag it to your Dropbox or Google Drive or wherever else you want to back up. That's an easy way to back up. And like I said, it also respects folder organization with inside the database too.

01:33:23:17 - 01:33:28:02
Robbie
So if you want to just take the whole thing and back it up, that's an easy, easy solve as well.

01:33:28:04 - 01:33:50:12
Joey
Yeah, and we've done that for various reasons of doing full back ups or backing up a set of drops or for archiving purposes. And what I usually do is I just bound a hotkey to export DRP. So once I'm like, cool, I'm making great project on the great progress on this project. Bang out a DRP to my default export folder, which is then syncs to a cloud file share just to have.

01:33:50:12 - 01:34:12:19
Joey
So I've just got a running DRP backup of everything that I'm doing. Just in case, you know, the internet goes down. I can just import that DRP and be back to work because again, our storage isn't reliant on the cloud. Our storage is sent to the three locations locally. So as long as I have that project information somewhere, if the entire internet goes down, I can still continue working.

01:34:12:19 - 01:34:30:00
Robbie
That's such a great point about the sync stuff back and forth is that, you know, I know both of us, but I'm going to make fun of you a little bit. You are about as data security, I'm going to say, conscious. But really what I mean is a paranoid.

01:34:30:05 - 01:34:31:15
Joey
Anxious.

01:34:31:17 - 01:34:51:02
Robbie
About losing a losing stuff more than anybody I possibly know. Folks, I'm not kidding you, like Joey, will back up things seven times and still be sweating at night thinking about losing data. So one of the added benefits, especially when you're you know, you're working on, you know, a big scan or something like that, that could take days to rebuild.

01:34:51:04 - 01:35:10:13
Robbie
Is that thinking does have that extra layer of security for us that, hey, if I lose everything, if my if my mask just blows up, guess what? I'm still thinking that to other places. And so it's kind of, you know, kind of inherently a little bit of a backup system, not a great one, but a little bit of a backup system because they're not going to last things.

01:35:10:13 - 01:35:32:08
Robbie
Before we wrap up here, we had mentioned some collaborating function gotchas. Now, specifically, what I mean by that is not the way that we've set things up, but collaborate mode inside of a of a project, you know, result project. When you're in collaboration mode, what are what are some of the gotchas that you've noted? I noted a couple, but I'm curious to hear your thoughts.

01:35:32:10 - 01:35:54:03
Joey
Yeah. So it is important to remember that collaboration is kind of its own thing and resolve. There's collaborative projects and non collaborative projects and you need to switch that on. And when you switch it on, it does have some implications. The biggest one is that it doesn't let you do dynamic project switching when you're in collaboration mode. So just be aware of that.

01:35:54:03 - 01:36:14:05
Joey
If you don't need to have a project set to collaboration, probably don't put it on collaboration because you lose that convenience of dynamic project switching. The way we usually do it is if we need to both be in the same project at the same time, we'll say, okay, I'm going to turn on collaboration. We both go into the project and then when it goes back to only one of us working on it.

01:36:14:05 - 01:36:37:04
Joey
Yeah, well, manually go ahead and turn off collaboration for that project because it does make some other things easier. The other thing that we've seen issues with is if you have an external map coming from the media pool into your node tree, we've had some odd issues only on collaborative projects where that doesn't link correctly across other machines.

01:36:37:04 - 01:36:40:05
Joey
And if you turn off collaboration.

01:36:40:08 - 01:36:41:00
Robbie
You know.

01:36:41:02 - 01:36:53:20
Joey
It will work. We had this there was a project a couple weeks ago where I was working on some keys while Robbie was working on some color and I'm like, Cool, Robbie, Keys are all done. They look great. We're ready to go. And he opens it up and he's like.

01:36:53:22 - 01:36:54:10
Robbie
What are you talking.

01:36:54:10 - 01:37:20:11
Joey
About? Trying to be polite about it. He's like, Are you sure they're all done? Are you sure you're great at this? Because on his end, something in the node tree wasn't communicating right over the collaboration mode and all of the keys and color looked completely nuts. Yep. While on my machines, fine. They were fine. So we backed out, turned off collaboration and went back in and it was fine.

01:37:20:14 - 01:37:45:16
Joey
So the only thing I would say is, one, we do know that the cloud part of this is still technically in beta. Yep. And two, the collaboration features are evolving pretty rapidly as well. So if you see something that's a little on the weirder side, don't panic. Think critically about it'd be like, okay, maybe this is a bug or a workflow problem on collaboration.

01:37:45:19 - 01:37:53:04
Joey
Let's try and troubleshoot it. Don't immediately think that Joey went blind and graded everything insane.

01:37:53:06 - 01:38:21:19
Robbie
Yeah, no, that's that's definitely a gotcha. I have noticed a similar a similar issue with the only way I can describe it is moving too fast. And so all the normal runs, all the normal rules about clip ownership, especially on the color page timeline apply with collaboration mode as they do locally. Right? So while I'm on that clip, I own that clip for you to see changes on that clip, I need to move off to the another one and then you need to refresh or whatever.

01:38:21:21 - 01:38:46:10
Robbie
I have noticed sometimes if I'm just, you know, next clip, next clip, previous clip, next clip, next node, whatever, bouncing around resolve in collaborative mode can sort of get a little confused if you give it a second. It usually sorts itself out, but sometimes that's an issue. And then the other one I've noticed that I'm sure it's just a matter of how the NWC logging works with resolve and how often it needs to re authenticate.

01:38:46:12 - 01:39:07:01
Robbie
I've noticed a couple of times I've logged in. Nothing seems to work like I'm toggling grades on and off. Doesn't seem to work and I'm not getting any warning from resolve. Right. I have found that nine times out of ten, if I quit resolve the next time that I launch it, it's asking me to re authenticate to the cloud database and just re log back in.

01:39:07:06 - 01:39:15:07
Robbie
Why it doesn't do that the first time when it doesn't want to respond, I don't know. But that seems to be the fix for most of those problems.

01:39:15:09 - 01:39:34:13
Joey
Yeah, and the good news here is you don't have to be too scared because the common thread in all of these kind of gotchas is that resolve isn't going to let you break the project if it doesn't have a good connection to the database. So like if you're doing what Robbie was saying and jumping back and forth between a clip really, really fast and for some reason it doesn't take ownership of the shot.

01:39:34:15 - 01:39:56:03
Joey
The color controls won't change anything, right? If you get logged out of the database and you try to delete a bunch of stuff out of the project, it won't let you because it doesn't have the permission to go to the cloud and say, Hey, delete all this stuff. So if you get into some of these kind of weird things that we've seen just back out, log in and come back in and your project will still be intact.

01:39:56:03 - 01:40:02:08
Joey
It's really, really good about doing what I think is the most important part, which is not let you break it.

01:40:02:08 - 01:40:21:12
Robbie
And I'm and this is not just like, you know, promotion for Black Magic and, you know, company, you know, you know, toting the company line or whatever. They're not doing anything for us. It's I honest to God truth in a year plus of using the cloud database, I have not lost a single project or lost any work or data from that.

01:40:21:14 - 01:40:33:17
Robbie
And when I've lost something, been like super stupid, normal, you know, color of screw up stuff like, you know, rippling something to 4000 grades and then, you know, messing that kind of thing up, Right?

01:40:33:17 - 01:40:53:22
Joey
Not because I've even had an issue where AWG went down, right? So our connection with the database just went away. We went down or my Internet connection went down and resolve will come up and say, Hey, I can't get to the database. Do you want to a export a DRP of your project because it keeps the project in memory.

01:40:53:22 - 01:41:00:21
Joey
So even if it disconnects from cloud when it errors out, it will let you export a DRP of that project to your local machine, which is a.

01:41:00:23 - 01:41:01:10
Robbie
Huge saving.

01:41:01:10 - 01:41:23:19
Joey
Awesome security blanket to have. But when that came up, when either my internet connection went down or when we went down, what I did is I just backed away from the system and waited for AWG to come back. And sure enough, the second it could see the database again, it was like nothing had happened. Actually, that's error went away and I just kept on working, just like my export to the backup.

01:41:23:23 - 01:41:24:10
Joey
And then I kept.

01:41:24:10 - 01:41:52:16
Robbie
Working, just like we spoke about a few minutes ago with the inherent data security of sinking media. But that is one last thing that I think we can wrap on, is that, you know, there's nothing saying that you can't have multiple layers of protection within your database, too. And so, you know, one thing that I think a lot of people are doing or thinking about doing is they're working locally in a local state, whether that's, you know, local SQL database and network SQL database or even a a project derived database.

01:41:52:18 - 01:42:05:09
Robbie
There's nothing saying that when you want to then share that with the rest of your teammates or whatever, you can't simply, you know, Commander control see Commander Control V and paste that into the cloud database, Right? It might.

01:42:05:09 - 01:42:17:01
Joey
Be. And you can do that across databases, right? And there was all project manager. You don't need to export and import anymore. In fact there's now a button in the top right to copy selected projects to any of your other databases.

01:42:17:01 - 01:42:35:23
Robbie
Exactly. So if you're the type of person that's like, you know, really antsy about other people in your facility, maybe, you know, looking at it, you know, and you're you're not ready yet or messing it up somehow, work locally and then copy that to the shared database when you need to do the rest of this workflow. So Joe, I think hopefully everybody kind of growing to what we're talking about here.

01:42:35:23 - 01:43:00:21
Robbie
We're, we're exceedingly happy with this workflow. And I think that if you guys have any questions, it we're happy to answer those. Again, not reinventing anything from scratch, but kind of putting these various pieces together. And it's allowed us to be incredibly efficient across multiple locations with multiple users with limited amount of headaches. And so, yeah, for the time being, that's how we're rolling.

01:43:00:21 - 01:43:12:11
Robbie
We're really happy with it and let us know if you adopt, you know, this in part or and all of it. And for sure, let us know if you have any questions. All right, guys. Well, that's it for this installment of the Offset podcast. I'm Robbie Carman

01:43:12:13 - 01:43:14:06
Joey
And I'm Joey D’Anna, thanks for listening.


Robbie Carman
Robbie Carman

Robbie is the managing colorist and CEO of DC Color. A guitar aficionado who’s never met a piece of gear he didn’t like.

Joey D'Anna
Joey D'Anna

Joey is lead colorist and CTO of DC Color. When he’s not in the color suite you’ll usually find him with a wrench in hand working on one of his classic cars or bikes


Stella Yrigoyen - Editor
Stella Yrigoyen

Stella Yrigoyen is an Austin, TX-based video editor specializing in documentary filmmaking. With a B.S. in Radio-Television-Film from UT Austin and over 7 years of editing experience, Stella possesses an in-depth understanding of the post-production pipeline. In the past year, she worked on Austin PBS series like 'Taco Mafia' and 'Chasing the Tide,' served as a Production Assistant on 'Austin City Limits,' and contributed to various post-production roles on other creatively and technically demanding projects.


Share This

Latest Episodes
EP004: AI & ML In Post

New Sponsor – Flanders Scientific! We’re thrilled to announce that The Offset Podcast now has a sponsor – Flanders Scientific! Flanders Scientific (FSI) is a leader in color-accurate monitoring solutions that are a perfect match for colorists, DITs, editors, and broadcast engineers. Their new XMP lineup features QD-OLED technology for truly no-compromise HDR monitoring. As … Continue reading “EP004: AI & ML In Post”

EP002: Assistants

Since this only our second episode, as a reminder before you jump into the episode, a few bits of housekeeping on listening, watching, and accessing some other features. 1.  An embedded audio player of an episode will always appear at the top of each episode page so you can listen directly in your browser. 2. … Continue reading “EP002: Assistants”

EP001: Client Communication

Before you jump into the episode, a few bits of housekeeping on listening, watching, and accessing some other features. 1.  An embedded audio player of an episode will always appear at the top of each episode page so you can listen directly in your browser. 2. Either by copying the URL of an episode or … Continue reading “EP001: Client Communication”


View All