EP004: AI & ML In Post
New Sponsor – Flanders Scientific!
We’re thrilled to announce that The Offset Podcast now has a sponsor – Flanders Scientific!
Flanders Scientific (FSI) is a leader in color-accurate monitoring solutions that are a perfect match for colorists, DITs, editors, and broadcast engineers. Their new XMP lineup features QD-OLED technology for truly no-compromise HDR monitoring. As FSI customers for over a decade, we truly believe their products are the best around. Head over to FSI today to learn more about their amazing monitoring solutions.
Robot Overlords or A Helping Hand?
Unless you’ve been living under a rock, you’ve likely heard the countless ongoing discussions about ‘Artificial intelligence’ and machine learning.
Before you dive in, please understand that we’re not experts in AI or ML – there are some extremely smart people (think multiple PhDs) who can discuss AI/ML related topics in far more detail than we can, however, we wanted to give you our perspective as informed end users.
While there are a lot of toolsets that can utilize machine learning (generically artificial intelligence) to do a wide variety of tasks – e.g. generative content creation, workflow enhancements, and addressing mundane tasks -does that mean the age of the creative operator is over? Are the jobs of colorists, editors, and other creative roles in postproduction dying because of AI/ML?
Maybe, maybe not.
In this episode of The Offset Podcast, Robbie & Joey share their thoughts on how AI/ML are currently affecting the postproduction industry and the future of AI/ML – both good and bad and how those things are likely to continue to affect creative roles in postproduction.
This is not a definitive discussion – and like we said there is so much we don’t explore in detail – but if you’ve been avoiding the subject, now is as good a time as any to start learning how AI/ML are influencing our industry.
Enjoy the episode!
-Robbie & Joey
Video
Links
- MIT Primer On Machine Learning
- In-depth MIT Articles on AI & ML
- IBM Overview of AI & ML
- Industry Thought Leader Katie Hinson's overview of AI & ML
Transcript
01:00:00:10 - 01:00:14:05
Joey
Hi. In this episode, we're going to talk about AI and machine learning and why the colorist is not dead yet. Stay tuned.
01:00:14:07 - 01:00:33:22
Robbie
This episode is sponsored by our friends Flanders Scientific, who are leaders in color-accurate display solutions for professional video. Whether you are a colorist, editor, DIT, or broadcast engineer, Flanders Scientific has a professional display solutions to meet your needs. Learn more at FlandersScientific.com
01:00:34:00 - 01:00:35:05
Joey
Hi, I'm Joey D’Anna.
01:00:35:10 - 01:01:04:07
Robbie
And I'm Robbie Carman. And guys, today on this episode, we're going to be talking about something that unless you've been living under a rock, you have been bombarded by, you know, traditional news, the internet, friends and family, and that is these buzzwords of artificial intelligence and machine learning and AI or AML for short. Now, just to be clear about something, Joey and I have opinions about AI and ML.
01:01:04:07 - 01:01:31:00
Robbie
Neither one of us are experts in AI or ML. We are not coding the platforms to do this. We don't know everything there is to know about them. There are doctorates, you know, people who have doctorates and who are, you know, PhDs or whatever out there who are doing this stuff. But we thought because, you know, in talking to our friends, our clients, our peers, there is so much buzz about AI and ML in our industry and post-production and color and finishing.
01:01:31:02 - 01:01:47:16
Robbie
You know, it runs the gamut from editing to, you know, music creation to whatever we thought to be good, just to kind of talk about this, share what we think are some of the fears that people have about ML and AI and why some of them probably are unfounded, but some of them might have little legs to them.
01:01:47:18 - 01:02:08:16
Robbie
And then kind of where we see things going for the future of artists and finishers and colorists like ourselves. So Joey, I think the first place that we should start is kind of the everyday layman's understanding of what is AI, what is machine learning? What do you think about those Those two phrases.
01:02:08:18 - 01:02:40:05
Joey
I have, as you may or may not know, very, very, very strong opinions on the pedantic terminology related to this emerging technology. In my mind, we are completely talking about a technology called machine learning. None of this is A.I., none of this is intelligent. The computer does not have any intelligence, the software does not have any intelligence. It is still algorithmically driven by code written by humans.
01:02:40:08 - 01:02:53:00
Joey
It is not thinking or creating. It is deriving based on a series of inputs, and it does not become artificial intelligence until it can rise up and try to kill me. And I have to fight it.
01:02:53:01 - 01:02:54:10
Robbie
Well, I can then you can.
01:02:54:13 - 01:02:58:19
Joey
Call it artificial intelligence. I make one. But right now we are talking about machine learning.
01:02:58:19 - 01:03:17:09
Robbie
And I make one note, though, that's concerning me is I've noticed that in your setup you have a Cyberdyne systems t shirt on and you have what appears to be maybe an arm from the Terminator. So I have a question for you. Based on what you just said about, you know, where there's no such thing really as A.I..
01:03:17:09 - 01:03:23:15
Robbie
We're all machine learning. Do you think, Hey, I will be a thing eventually?
01:03:23:16 - 01:03:30:10
Joey
I hope not. I absolutely hope not. But if it does, I'll be ready.
01:03:30:12 - 01:03:46:01
Robbie
So I think it's a great distinction that you made. I think a lot of people use artificial intelligence because it sounds a whole lot cooler than machine learning. Machine learning sounds like a certificate that you'd get at a community college to like, you know, operate. Yeah. Operate a forklift or something. Right.
01:03:46:02 - 01:04:03:02
Joey
But like the Terminator, also artificial intelligence sounds a lot scarier than it is because people are terrified of losing their jobs about having the evil robots take over all the industry. And that's just not what this AI technology does. It has no actual intelligence. Well.
01:04:03:04 - 01:04:04:01
Robbie
I think that's a good.
01:04:04:01 - 01:04:04:18
Joey
Don't be scared.
01:04:04:18 - 01:04:27:06
Robbie
I think that's a good place to start because, you know, just to give people a little peek of of our history, you know, both of us started in post-production in the mid to late nineties when there was a lot of heavy iron, literally heavy iron machinery going on in in and machine rooms, you know, big linear rooms, lots of tape decks, that kind of stuff.
01:04:27:06 - 01:04:48:01
Robbie
And we've seen a, you know, an evolution, of course, in post-production color, audio motion graphics. I mean, the idea that, you know, you could set, you know, have an application to fly text around by setting key frames on, you know, an app used to be mythology back in the day when people were running, you know, big, expensive Henrys and Karens and all sorts of stuff.
01:04:48:03 - 01:05:30:23
Robbie
So the idea I think that I want to address first is this idea of technology evolving and changing. Right? And I think naturally that's scary to people. And that's the thing I think about most actually, is to kind of whirlwind changes in our industry. Number one, the jump to the desktop editorial systems, right, moving from million dollar, you know, C-Max, Grass Valley, Sony, whatever, you know, linear rooms to people going, no, this is my Mac with the Avid on it and later media 100 video toaster You know whatever it may be you know final cut pro those kind of things and going I'm going to lose my job as an editor because I'm working
01:05:30:23 - 01:05:35:20
Robbie
on this million dollar editing suite and now it's on a computer. Like what? That didn't happen, right?
01:05:36:00 - 01:05:39:12
Joey
People thought Final Cut Pro was going to bankrupt the entire post-production industry.
01:05:39:17 - 01:05:40:05
Robbie
It didn't.
01:05:40:05 - 01:05:41:10
Joey
It just made it bigger.
01:05:41:10 - 01:06:07:03
Robbie
It just made it bigger, faster, more efficient, more. You know, this was bandied about a lot, but sort of democratized that toolset to more creators being possible there. And I think that, you know, we've seen a lot of transitions like that, that number one. And number two, I think the other transition that comes to mind is people being all sorts of freaked out and scared about was when we made the transition from tape based workflows to file based workflows.
01:06:07:03 - 01:06:25:02
Robbie
Right? People were, you know, I mean, yeah, unfortunately places like dubbing houses and places like that who didn't change probably did go out of business. But in general, I think everybody can say, Hey, a file based workflow is a whole lot easier than managing, you know, a fleet of tape decks and the expense of those tapes and making stuff.
01:06:25:04 - 01:06:31:12
Robbie
So change like this in my opinion, is not anything new to our industry. And I actually get.
01:06:31:14 - 01:06:32:00
Joey
I get a.
01:06:32:00 - 01:06:42:21
Robbie
Little worked up when I see news reports on this stuff because in my opinion, it's kind of like fear mongering to a certain degree, right? Like be afraid you're going to lose your job. And I think it's.
01:06:42:21 - 01:06:44:06
Joey
An easy way to get clicks.
01:06:44:06 - 01:07:04:11
Robbie
Absolutely. And I think it's in responsible of a lot of, you know, newsmakers to do that because, you know, that fear mongering never suits anybody except for ratings and getting clicks, etc., when the reality of it is the majority of us will integrate and evolve this kind of technology into our own workflows without having to be kind of, you know, scared of it.
01:07:04:11 - 01:07:35:23
Robbie
So I think that's one point I want to make. The next point I want to make, and I think I want to your opinion about this because it's a little confusing to people besides the two terms email. I there's also kind of this difference in what Emily does, right? I think a lot of the the news that we see, the hype, the buzz is about generative AI and that is AI that is kind of seemingly aware of what it's doing.
01:07:35:23 - 01:07:52:16
Robbie
And, you know, as you say, it's not really it's just using programmatic approaches to it, but, you know, it's creating art, for example. Right? Is that real art or is that not real art? It's doing things like, you know, the new Photoshop ability to like, hey, I need to fill this area, generate something that would look cool back there, right?
01:07:52:18 - 01:08:09:10
Robbie
That generative AI is one thing. What are your feelings about generative AI or machine learning things? You know, this these tools that create something, is that a good thing? Is a bad thing? Do you think it has a place in post-production and finishing for us, even if that's right, You know, right this moment.
01:08:09:12 - 01:08:36:01
Joey
Yeah. I mean, it's definitely the most kind of eye catching, interesting application of this kind of machine learning technology, right? It's the it's the wow factor. It's like a computer made that image that looks photorealistic just based on a text description. That is amazing. But at its core, what it's doing is it's basically taking a gigantic library of learned information.
01:08:36:01 - 01:09:14:23
Joey
Right. It's been fed all kinds of still images, video, corresponding descriptions, words, things like that, just a gigantic set of existing data. Right. And then the parameters are basically going through a bajillion if then else statements and kind of figuring out step by step how to make what you asked for in a effective and convincing way. Now, where this breaks down in terms of calling it something like creative or creating is all it can ever do is derive from what it's been given.
01:09:14:23 - 01:09:36:19
Joey
So if you feed it a crap ton of good photography, it might come out strong. That looks like good photography. You feed the crap crapped on of bad photography. It's probably going to come out looking like a little bit of a worse photographer if you try to ask it to make a photo. If you feed it all of one ecosystem of things, everything that's going to come out of it is from that ecosystem.
01:09:36:19 - 01:10:01:08
Joey
So it's like anything else, what you get out of it is going to depend on what you get into it and what's really interesting for me, for our industry is not the technical part of that, right? It's kind of the legal, creative, moral part of that because and there's been a lot of talk about this and right now there's not really a regulative framework or even a best practices framework for this.
01:10:01:10 - 01:10:27:02
Joey
A lot of these generative eyes are trained on copyrighted original art without the full knowledge and understanding of those artists. So essentially you're just asking a computer to pirate something, copy it and wiggle it around into a more different set of pixels. Now that might be the worst case kind of explanation. Let's also talk about kind of the really good case explanation of this.
01:10:27:04 - 01:10:49:01
Joey
Are companies like Blackmagic and Adobe that are introducing machine learning tools, obviously have very big legal departments and have very good concern about these legal and moral issues. And they've all basically said we don't use any user generated content, we don't use any of your content to train our algorithms, and we don't just go out and scrape the Internet to train our algorithms.
01:10:49:01 - 01:11:11:09
Joey
They're much more targeted and that does two things. One, like I said, if you put in bad stuff, your results will suffer. So they're curating the inputs to their algorithms with very careful things to make it do what they want and they're making sure not to kind of steal people's copyright. Now, in that case, the generative AI, in my opinion, has a couple of really good uses.
01:11:11:09 - 01:11:33:22
Joey
One, like you said, the kind of Photoshop fill something in object removal in in video applications, for example, it would need to know what does grass look like to remove something from grass? And that's the kind of thing where it's a it's not really an artistic decision, right? It's not like the entire creative intent of this shot is removing this shadow from this grass, right?
01:11:33:22 - 01:12:03:10
Joey
So the execution of that image operation doesn't really have a drastic amount of creativity in it. And it's a great thing for the robot to handle and save you some time. The other thing that I think is really interesting is if you go back to the idea that I said of machine learning, taking this gigantic pile of visual data, this gigantic dataset, and then generating something from it, like I said, it's not making art, but it's making an image derive from its set of inputs.
01:12:03:12 - 01:12:40:03
Joey
It can be a fantastic tool for inspiration and brainstorming. Try throwing different things at it and seeing what it comes up with. It has a way more diverse source of information than your single life experience or your single life experience as an artist. So using a generative A.I. tool, I got, I shouldn't say that a generative machine learning tool, even I'm guilty of breaking my own terminology rules sometimes, but using those generative tools for inspiration, finding themes, colors, looks, ideas that you might not have had, and brainstorming that that democratizes the creative process, right?
01:12:40:07 - 01:12:57:00
Joey
It lets somebody who hasn't traveled the world see things and inspire their work with all of this different media that has been trained into these algorithms. So that I think is like, that's a really exciting use case for generative machine learning.
01:12:57:04 - 01:13:19:06
Robbie
Well, there's a lot to unpack there, and I think I agree with almost everything you said. I will say I think there is some debate to whether or not how should I say this bluntly, that is, is there art to coding rape? And I think that there can be. And I was trying to my kids were asking about, you know, generative AI because you know, somebody at an art school or whatever.
01:13:19:12 - 01:13:24:20
Robbie
Somebody had mentioned MIT journey and they were looking into it. How does it work and that kind of stuff. And it dawned on.
01:13:24:20 - 01:13:30:11
Joey
Me there's also some creative intent to the prompt you give it and how you word it. So that exact completely devoid of art.
01:13:30:12 - 01:13:51:06
Robbie
Exactly. And it seemed to me when I started playing with it in hopes of finding better uses and or explanations for my kids, I kind of was hit upon something, and I've had the same experience using, you know, chatting up the other day I asked Chad, she said, you know, write me a song, song lyrics about X, Y, Z, Right?
01:13:51:08 - 01:14:11:00
Robbie
And I tried it a few times with different additives, different, you know, changes to the way I phrase the the phrase, the request, that kind of stuff. And it dawned on me that part of the game, if you will, of generative AI is taking what is under the hood. I think you're absolutely right. It's a series of logic.
01:14:11:00 - 01:14:35:10
Robbie
It's a series of program, programmatic steps that scours different places to figure out what you're talking about, how you're talking about what you want it to look like, etc. Very complex stuff that I mentioned at the top of the show. PhDs are involved in, and it's beyond me, right? But what I realized about the project part about it is that we're taking really programing to the truest whizzy wig approach that we possibly can right?
01:14:35:10 - 01:14:56:03
Robbie
We're saying just say something naturally in human language, the way you would describe it however you want, and we'll create something like that. And it's very much a way of, in my thinking, democratizing programing to a certain degree, because you're instead of typing in all the lines of code, you're just saying, Hey, I want this, right?
01:14:56:05 - 01:15:24:05
Joey
Yeah. And I think that's actually an excellent, excellent point because we've seen in the history of computing a lot of different attempts to do that from like basic and logo and things like that where they're much more plain language programing languages. I've always said that, you know, a computer will only do what you tell it to do. And these tools are allowing people to tell computers what to do in a clearer, more concise, more digestible for them to play.
01:15:24:05 - 01:15:25:14
Joey
And that's a good thing.
01:15:25:16 - 01:15:51:08
Robbie
But then my next thought after that was, Wow, this is very similar. My interaction with MIT Journey or Chhatrapati your, you know, the the Google tools or whatever it may be, the platform, maybe the Adobe Tools was this is very much the same situation that I'm faced with. You're faced with and probably a lot of our listeners are faced with when interacting with clients, describing what they want to see happen on screen and for you to change, right?
01:15:51:13 - 01:16:13:10
Robbie
In other words, yeah, we are often the AI machine learning algorithm and our clients are the person putting in that prompt. Right? And you know, it's hard to know nauseum about, client communication and this is, you know, how to interpret this. And people make jokes about us being post-production psychologists and, you know, translating what our clients say.
01:16:13:14 - 01:16:48:09
Robbie
And it dawned on me when I was starting this this experiment with these tools, it's the same thing, man. It's only going to be as good as the way I describe it and what my needs are out of it. Right? And a part of that is wordsmithing and, you know, kind of being descriptive. Part of that is learning the way that's algorithms want and accept prompts and modifications like, you know, you get big results, big result differences out of mid journey by, you know, clicking different, not clicking but in sort of enabling different, you know, switches and and such to do different levels of quality and you know how long it takes and that kind
01:16:48:09 - 01:17:08:22
Robbie
of stuff. And so that was one thing that got me about this was that like you know, I think a lot of people who are blog on this might not be the most descriptive people in the world, right? Their experiences with it are going to be different than somebody who's very verbose, very detailed, that kind of thing, versus somebody who just goes, make me a flower.
01:17:09:01 - 01:17:21:09
Robbie
Right? You know, there's a big difference between making me a flower and make me a flower that has subtle shades of pink, reds and blue with dew dripping on it with morning sunlight, you know, whatever you want to throw at it.
01:17:21:09 - 01:17:24:14
Joey
Shot with an 85 millimeter lens two feet away, you know?
01:17:24:19 - 01:17:49:12
Robbie
Exactly. Exactly. So, you know, the people, I think, who are having bad experiences with generative A.I. may not be the most descriptive people. But the other thing I thought about with those people who are having trouble sort of walking with generative A.I. does, is that I think it it takes a while to, in this sense, learn the literal vocabulary of what these tools can do.
01:17:49:14 - 01:18:10:19
Robbie
Just in the same way that we've talked. You know, we're working on an HDR project right now. And you know, earlier you said to me, I think you pushed it too far and we've talked a lot about like the vocabulary of what makes good HDR grades and not it seems similar here, right? It seems like there is there's a scale to this, like the vocabulary of how we interact with these tools is, is getting better and refined and different.
01:18:10:19 - 01:18:32:08
Robbie
And in fact the people making the tools are going, well, you know, we need to take into account when there are one word that could mean seven different things. Right. And how they program that vocabulary and all that kind of stuff. Right. So I think it's there for me though, Joey, I think generative A.I. in what we do is right now all of it at its infancy.
01:18:32:08 - 01:18:52:11
Robbie
But for a practical standpoint, it's really hard to do in video work right now. I think it's I think it's awesome for making a still like my daughter had her 15th birthday. She loves Taylor Swift. Sure I'll go on mid journey and go create me you know a picture of my daughter on stage with Taylor Swift with 20,000 adoring fans.
01:18:52:11 - 01:18:58:01
Robbie
And I can do that. And it looks cool, right? And looks like my daughter was on stage with Taylor Swift, you know, at a concert right now.
01:18:58:01 - 01:18:59:21
Joey
Again, there's copyright implications there.
01:18:59:23 - 01:19:21:22
Robbie
That's true. But but, you know, that kind of thing relatively easy to create when I'm going sitting in the street with a client in the sciences, you know, I'd really love to have a drone shot coming over the Hollywood Hills. Hills that reveals Los Angeles with a thin layer of smog at dawn. And you know what I want that to be?
01:19:22:00 - 01:19:45:00
Robbie
I want it to be very filmic. So I want a, you know, shallow depth of field like that kind of stuff, way more difficult to create in motion that's going to fit in. Right. And I think that's one of the reasons that a lot of us are struggling with these concepts right now, because we're not actually seeing it yet totally in a place that is like, I can make a 1 to 1 connection and how that's going to work with my work.
01:19:45:02 - 01:20:10:05
Joey
Yeah. And you know, it's always going to be like that. I don't want to be the naysayer that says, this will never be perfect because, you know, technology does move in very dramatic steps. But at the end of the day, this is generative. A.I. is never going to replace a photographer or a cinematographer for original content or an animator or an illustrator.
01:20:10:10 - 01:20:20:09
Joey
It might help them along as a tool. They might utilize it, but it's never going to have value as just full origination.
01:20:20:14 - 01:20:41:00
Robbie
That's a great point and that's kind of where I like. So I'm thinking about as a colorist and what I do every day. Where would I want General have I to step in? And I can think of just off the top of my head, kind of three or four things that I wanted to do that tools exist to do this, but they're not, you know, tell it to do it kind of thing, right?
01:20:41:02 - 01:21:00:04
Robbie
So number one, I think it is the the most obvious one is fill in object removal. Right. You know, just be like whether it's a sky replacement, whether it's hey, remove that lamp from the background and fill it with part of the table that looks natural without having to do all the roto work and all that. So I think I think that's pretty simple.
01:21:00:06 - 01:21:23:02
Robbie
I think too, it's about to me, it's about lights. I would love to be able to go, Hey, I tool a machine learning tool. I would like to have a tungsten light, you know, a diffused tungsten light on the right part of this frame and not have into me it would be perfect is not to have it, just do it, but then to present.
01:21:23:04 - 01:21:36:19
Robbie
Okay, here are a level. Here's a level of control that you have over the further refinement. I've done that. I've done the heavy lift for you to make it happen. But now you're the artist. I'm just a stupid computer program. Now you refine.
01:21:36:19 - 01:21:59:03
Joey
That to me. Is the difference between a usable professional tool and a toy, right? The straight text in image out is a toy or a professional tool for finding inspirations that you then will throw out and then do something on your own. Having the ability to go in and change it afterwards, which is a very tough challenge for the developers, right?
01:21:59:03 - 01:22:20:06
Joey
Because you don't have a lot of flexibility as to what these systems output. So having the ability to go in and customize after the fact and refine without losing the bulk of the work that has been done, that is the absolute key killer aspect of this that we need for any kind of machine learning tools if they're going to be used in professional post-production.
01:22:20:07 - 01:22:47:07
Robbie
I had a great talk a couple of weeks ago with a friend of mine who is I mean, deep, deep, deep into the world of Unreal Engine three. And his job has been creating photorealistic imagery that will go on LED volume walls. Right. You know, so if you know, whatever you're filming the show and somebody's supposed to be on a on a ship at sea, he'll do all that stuff in Unreal Engine and they'll put it on the little volume wall.
01:22:47:07 - 01:23:08:18
Robbie
They'll film in front of it. And nobody knows that it wasn't practical, really. His take on this, I thought, was pretty interesting that he thought generative A.I. in, you know, again he's using one platform, Unreal Engine, which is popular for this kind of thing. But his take on it was partly inspiration, like you said, but partly also just to get to the result quicker.
01:23:08:18 - 01:23:29:00
Robbie
Right. And, you know, he's making the point, if I can design you know, I can design a mountain landscape from scratch, but it would take me a while to do that. If I can integrate generative AI into it and say, make a mountain landscape, but then go back in with the rest of my toolset, finesse it, finagle it, whatever.
01:23:29:02 - 01:23:35:14
Robbie
That's what he's looking for. And I thought that was a wise, wise thing, as you said, because that's what makes it not a toy being able to go, Yeah.
01:23:35:18 - 01:23:43:22
Joey
I can handle the heavy lifting of stuff that's in the background. That might not be the direct focus, but still needs to be there.
01:23:44:00 - 01:24:07:19
Robbie
Absolutely. So I think generally by we're going to see that constantly developing. I think the hybrid thing, like I just said with Unreal and I was that's there and I think it will get better right now. I think in practical terms for what we do in post-production and production, I think what gets dismissed a lot is and you say this to me all the time as I'm clicking like 39 buttons, right?
01:24:07:19 - 01:24:32:23
Robbie
You go, Why wouldn't you just write a script for that? Right? But you hit it. You when you say stuff like that to me, you hit on a very valid thing about what we do as professionals day in, day out. And that is repetition, right? I can't tell you how many times a day I click on the same button or how many times a day I make the same set of exports or files for a different project.
01:24:32:23 - 01:24:54:12
Robbie
Right? So I think one of the places that I'm curious to hear your thoughts on this because your Mr. Efficiency, you know, is where machine learning and AI can help us with repair additive task because, you know, I just envision a situation the final thing comes to mind when we're finishing a show and we have 74 deliverables to make right?
01:24:54:14 - 01:25:14:00
Robbie
I would love a situation where one day deep into resolve goes, just promise me and says, Hey, Rob, I've noticed that you're on the deliver page. Last time you made all these exports. You want to do that same thing again without me having to go and click and make all the choices or set up presets or whatever, it just kind of intuitively knows where I am.
01:25:14:00 - 01:25:31:15
Robbie
The process based on what I've done that precedes where I am at now and then intelligently prompts me to do something just kind of like, you know, my Amazon Alexa will go, Hey, I notice that you're out of shaving cream. Would you like me to order some more? You know, like, that kind of thing? But taking up the next level.
01:25:31:17 - 01:25:59:23
Joey
Yeah. And to me, that's that sort of efficiency booster is, in my opinion, the absolute best use case for machine learning technology in not only post-production, but in any professional industry, because repetitive tasks are wasted time and wasted time is wasted money. Right? When Ford makes 50,000 Mustangs a year, they have a robot do the same welds on every chassis every time and put the same bolt in the same places.
01:26:00:03 - 01:26:28:15
Joey
And a lot of work goes into making those efficiencies. So where things can be repeated, where you don't need human input, it's all automatic. And like you said, any time it's like I'm doing this 1000 times, just write a script. That's always my answer and to a fault, because I'll if I find myself doing anything that's repetitive, my general rule of thumb is like if I feel like I'm working too hard for this, I probably am and something should be optimizing it.
01:26:28:17 - 01:26:50:12
Joey
I found myself go down ridiculous Rabbit holes where I will spend like 8 hours trying to write an incredibly elaborate script to automate a simple task to save me an hour of time. Right now. But part of that is my own kind of emotional attachment to not doing repetitive, boring tasks and trying to find an interesting solution to that problem.
01:26:50:12 - 01:27:05:13
Joey
At the same time, I enjoy that, but part of it is also really saving time and the value judgment is, well, is this going to be a set of repetitive tasks that I also have to do again, in the same context, If it is and I have to do it often, then yeah, it's worth the 8 hours to write the script.
01:27:05:13 - 01:27:30:04
Joey
If it's a one off, probably not. Right? Yeah. And those little cases, those one offs or where this kind of machine learning stuff could really, really, really hugely be useful. Like you said, democratize the programing. If I could type into a prompt in resolve. Hey, take 5 seconds off of every gap in this timeline. Those are pretty simple instructions, right?
01:27:30:04 - 01:27:48:12
Joey
It could parse what a gap is. Find all the gaps each past 5 seconds is take those gaps out. Right? That would be a manual process that I could easily automate just with a prompt and that kind of potential, like you said, also having it kind of keep an eye on what you're doing and look for patterns and find things.
01:27:48:14 - 01:28:08:17
Joey
I envision it taking that even to the next level, kind of like God, what Microsoft tried to do so many years ago with the Stupid Paperclip and Microsoft Office like, it looks like you're writing a resume. Would you like a template? No, but it looks like you're rendering a PBS show. Would you like us to pull up the specs that you used for the last one?
01:28:08:17 - 01:28:10:21
Robbie
Absolutely. I mean, I'm already we see.
01:28:10:21 - 01:28:28:21
Joey
Or even more so here's another one. It looks like you probably wanted to render this as Progress HQ, not regular progress, but are you sure? Because you did everything else in this project as HQ and you might have just clicked and that'll save me a re if I know if it spots me screwing up.
01:28:28:22 - 01:28:50:19
Robbie
Dude, I cannot tell you how many times that little bit of machine learning programing in for example in Gmail has saved my butt, not just grammatically and what I've said, but also saved me so much time. I'll type something and be like it's, you know, tabbed to fill in the rest of the sentence, right. Like that autofill kind of thing.
01:28:51:00 - 01:29:02:08
Robbie
And it seems stupid, right? I do feel like that's not the same thing as it is absolutely the same thing as me. It's figured out in a predictive way what you're doing, what you want to say, how are you going to say it based on your words?
01:29:02:08 - 01:29:33:13
Joey
You know what's funny? I was writing an email to someone that was, Let's just say, quite terse. Okay. And Microsoft Outlook and it's new modern machine learning spellcheck thing literally came up and said, Are you sure you want to use that tone? And gave me suggestions for not being so like mean in my email. Now in that email I decided to just be mean and ignore it, but I thought that was really, really interesting.
01:29:33:17 - 01:29:36:22
Joey
Yeah. And it was right. I was writing a mean email.
01:29:37:03 - 01:29:56:13
Robbie
And I think that the approach is so far to what we do, rightly so, have largely focused on the creative aspects of what we do. Right. So there's plenty of tools out there now that are doing MLA. I, you know, shot matching, for example, right? Figuring out the tonality of one shot or a reference and matching it to another.
01:29:56:15 - 01:30:12:08
Robbie
It's cool. Saves a lot of time because a lot of the time what we do is we're trying to, we're trying to match stuff. There's a lot of tools that will do things based on ID, right? So even in resolve, like the Faces feature, right? Hey, just create some bins for me automatically based on who these people are.
01:30:12:08 - 01:30:16:15
Robbie
Right. Super useful. I think those kind of tools to getting.
01:30:16:15 - 01:30:23:19
Joey
Reached a text that works totally. Yeah. We've had speech to text for 20 years. It's never worked until machine learning technology.
01:30:23:21 - 01:30:49:10
Robbie
And I think that the programmers are right to focus on those workflow enhancements that make editorial color, all that kind of stuff better. But I think there's still a really large workflow gap right now, honestly, like that exists for those repetitive tasks that are the next level up from things you can do with, you know, droplet or drop folders or even basic scripts, right?
01:30:49:12 - 01:31:11:09
Robbie
Like the one that, you know, you and I talk about all the time, especially when we have difficult conforms is I would like and you know a machine learning tool that I can just say okay man, here's a folder of media, here's the reference I got from the client. Go figure it out. Do it, go figure it out, piece, piece it all together.
01:31:11:09 - 01:31:13:15
Joey
And then tell me what you couldn't figure out. Right?
01:31:13:15 - 01:31:28:08
Robbie
And it doesn't necessarily have to be based on what we've always thought it, you know, prime code, real number, etc.. It goes, No, no, no. The shot didn't have a time code. It didn't have a real number. I just looked at the shot, right? I looked at the shot and I figured out how to match those two up.
01:31:28:10 - 01:31:58:19
Joey
Have it. I matched sizing to a reference bullet. We're already seeing this a little bit with. There are some tools to say, Hey, what is that font? Yeah, which is really useful. Yeah. You know, so seeing those kind of workflow tools evolve, I think, you know, that brings me back to kind of the original topic of this episode is the Color is Dead, Is the machine learning robot going to take away our jobs and are the studios going to say, you know what, Robot T-800 is going to grade this next series?
01:31:59:00 - 01:32:18:08
Joey
No, absolutely not. But it's going to make our jobs faster and it's going to let us spend more time iterating the creative. That's that's my key here is like, okay, if we can have the machine learning tool that fixes our conform or helps us with our conform and builds masks for us and helps us make deliverables, like you said, right.
01:32:18:10 - 01:32:39:12
Joey
I don't want to do an eight hour project in 4 hours now, right? I want to do the same eight hour project with 4 hours less administrative nonsense and four more hours focusing on iterating the creative. Because the more passes you do, we always work in passes, and the more passes you do, the better it gets until you run out of time.
01:32:39:15 - 01:32:45:02
Joey
Right? So this just gives us more time to be creative. And that is what I'm excited about.
01:32:45:03 - 01:33:00:17
Robbie
And to flash back to what we I said at the beginning of this episode about how people you know, there were lots of periods of time where people were afraid of that technology change right. And what were they really afraid of? They were not. It wasn't. Yeah, sure. At some levels I'm going to lose my job, but I'm going to be out of a job, Right.
01:33:00:19 - 01:33:39:19
Robbie
Really, what a lot of that was about was I am stable and comfortable with my knowledge base and where I am right now, and I'm scared about learning new stuff and how to integrate that into what I do. And so this, just like anything else, I think, is the people who are going to be most successful are the people who are interdisciplinary and about this kind of stuff, learning about generative and repetitive tools in machine learning and how to integrate that stuff best into their workflows, I can see a time where, you know the best, you know, colorists and grinders and facilities in the world, they're the best because they also have some of the
01:33:39:19 - 01:33:58:10
Robbie
best generative tools and they have some of the best repetitive tools or whatever. Just like we saw a huge leap when, you know, in 3D animation with new tools like Maya and those render, you know, render main engines and that kind of stuff, that technology helped that industry do things that were better. And I think the same thing is true with us.
01:33:58:10 - 01:34:17:07
Robbie
I mean, I would just love that. You know, the other one I think about all the time is going back to some of the machine learning stuff that I think is just there. I would love to be able to say set skin tone, you know, set the look and set skin tone on somebody and go, hey, look at the rest of this group.
01:34:17:09 - 01:34:18:01
Joey
Keep it going.
01:34:18:06 - 01:34:21:02
Robbie
Keep it going. Like make sure made the creative decision.
01:34:21:07 - 01:34:24:10
Joey
Yeah, keep it going for me. I'll I'll I'll come back and check in on you when you're done.
01:34:24:10 - 01:34:54:18
Robbie
And screw screw Rob screwing up repo for the hundredth time this week and just let the machine learning figure out how to repo that change to wherever and yourself. And better yet I think you know I think Samir a big part of this right you know if I make a decision without the help of the system. Right. I go, well, I'm going to make this person a little more olive complex, whatever, you know, the system going, Hey, look, I've looked at all these shots and I think that you're making the wrong choice somehow, right?
01:34:54:23 - 01:35:02:04
Robbie
You know, like, you know, or whatever, some sort of prompt by saying, Are you sure you want to change this? Because this seems crazy. That goes back to the.
01:35:02:04 - 01:35:24:06
Joey
Core concept of what this is, which is a pile of information and a prompt or some user input put together in a very complex web of programmatic give statements and where that can be super useful, like we've been talking about, you know, post-production software is if part of that information is the functionality of the software. That's what we haven't seen yet, right?
01:35:24:06 - 01:35:45:14
Joey
We've seen individual tools where it's like, okay, we're going to make a depth map. That's what it does. We're going to make a matt. That's what it does. But there's no tool that knows all of the different things that resolve can do and then also knows how to interpret English. So you could say put in a prompt, Hey, add every timeline to the render cue for me.
01:35:45:16 - 01:36:02:04
Joey
It knows what the render cue is. It knows the timeline. Is it know what's the word ad means in this cloud of inputs. It has all it needs to save you time. That's the kind of workflow tool that I think could be really, really cool. It's never going to do everything. But like just to have it, I think it would be awesome.
01:36:02:05 - 01:36:20:06
Robbie
Yeah. And I think the last thing that I'll rob my last thought on this is I think you brought up something that's above and beyond our pay grade pay grade knowledge, but I think is really developing fast and interesting to see how smart people figured this out is kind of the ethical and legal concerns about some of this stuff.
01:36:20:06 - 01:36:44:08
Robbie
Because one of the things that I'm always thinking about when I think about this subject is, as you said several times, the tools are only as good as what they're being fed. Right. You know, we did a project a number of years ago where it was colorizing World War Two, black and white footage. And at the time, you know, it was a really heavy lift with machine learning, you know, giving you as many war two images as possible to generate maps.
01:36:44:10 - 01:37:02:18
Robbie
And it was only as good as what we were giving it. Right. And we had to, you know, scour for thousands and thousands of images. I think part of the ethical and legal part of this is that the powers that be need to figure out a balance between privacy, copyright and also feeding those engines in a way, because, you know, if.
01:37:02:18 - 01:37:09:19
Joey
There was transparency, so you know what's going into what it is and if your stuff is being used and you agree to the context in.
01:37:09:20 - 01:37:26:17
Robbie
A way of pulling it out if necessary. Right. Like if you accident, if you accidentally, you know, hit scan my timeline for information and it's a you know, music video that, you know, is not released yet and the artist is going to be pissed by that like, you know yeah don't share it. I think there's that balance is yet to be found.
01:37:26:19 - 01:37:52:13
Robbie
But I'm curious as the stuff works through these ethical and legal concerns, how that kind of stuff works out, because I think the data the data is unbelievably massive in there. And, you know, right now we're going with a slightly smaller, you know, hopefully, you know, sort of control approach to getting some of that stuff. But the floodgates will be open when everybody's stuff is just there.
01:37:52:15 - 01:37:54:08
Robbie
But I just want to do that in a smart way.
01:37:54:10 - 01:38:29:18
Joey
Yeah. And in the same vein of the ethical application, I think it's really, really, really, really important to make a distinguishing comparison between what a generative fix of an image or a generative improvement of an image is in context of reality, specifically with regard to documentaries and historical and stuff like that. I have I won't say problems, but big questions and implications to think about with a lot of these machine learning based, for example, rescaling tools.
01:38:29:23 - 01:38:57:20
Joey
Okay, if you're doing a historical documentary and you are scaling up a piece of film using a generative A.I. tool, you are adding in information from the modern world that was not there. You are asking your computer to take piles of modern images and make something approximate to what this was. But you have now in some way devalued the historical accuracy of that point.
01:38:58:00 - 01:39:11:13
Joey
And it's the same thing with colorization. It's the same thing with machine learning based like sharpening and noise reduction. Those details that are being put in are fabricated. They aren't real. They were never captured in the original caption.
01:39:11:15 - 01:39:12:08
Robbie
Again, I'm not.
01:39:12:08 - 01:39:25:16
Joey
Saying there's no use that and it should never be used. I'm saying documentarians should be very aware of what the technology is doing and the implication to the historical accuracy of archived images.
01:39:25:16 - 01:39:38:20
Robbie
And it gets worse from there. Of course not. We don't have time for this today, but like I mean, already there is the ethical conundrum of deepfakes and you know how that stuff works in kind of creating news and, you know, putting more.
01:39:38:21 - 01:39:54:03
Joey
But those are so easy to pick out and it's so easy to write a program to pick them out. What gets me is like if I'm doing a documentary and I've got some old film and we scale it up using the best machine learning basically. Wow. You can see the detail in his face. Well, that's detail that was never captured.
01:39:54:05 - 01:40:16:05
Joey
We've just made up detail in that guy's face. Is that the right thing to do in a documentary? Only the documentarian can answer that question, but they should be armed with the knowledge to ask that question. I think in a lot of cases people might be excited by this new technology, think it does magical things and not realize what's going into it, and the kind of ethical honesty implications of that.
01:40:16:07 - 01:40:34:10
Robbie
So the colorist is dead. Long live the colorist is what I'm going to say. I think that, you know, it's always been the case that those who adapt and learn are going to be at the top of the heap versus those who, you know, grumble about it and don't want anything to do with it and refuse to learn.
01:40:34:10 - 01:40:55:22
Robbie
So I would urge everybody, you know, again, Joey and I are not experts. We don't have PhDs in machine learning in AI. This is just our opinions as users. But I think one, if I look at it as an exciting time to be in, certainly interesting for a lot of legal, ethical, whatever you know reasons that again, some people will hopefully apply and figure out.
01:40:56:00 - 01:41:16:01
Robbie
But I think us as users, whether it's repetitive, you know, repetitive fixes, whether it's, you know, you know, using generative stuff in a controlled way, whether that's inspiration or fixing problems, I think we're just at the beginning there. And I think that it pays for all of us to kind of, you know, do our research, think about this and try it out.
01:41:16:03 - 01:41:23:21
Robbie
So plenty to think about with all this stuff. Joey I think it's been a good talk. And for the Offset podcast, I'm Robbie Carman.
01:41:23:23 - 01:41:25:15
Joey
And I'm Joey D’Anna - Thanks for listening.
Robbie Carman
Robbie is the managing colorist and CEO of DC Color. A guitar aficionado who’s never met a piece of gear he didn’t like.
Joey D'Anna
Joey is lead colorist and CTO of DC Color. When he’s not in the color suite you’ll usually find him with a wrench in hand working on one of his classic cars or bikes
Stella Yrigoyen
Stella Yrigoyen is an Austin, TX-based video editor specializing in documentary filmmaking. With a B.S. in Radio-Television-Film from UT Austin and over 7 years of editing experience, Stella possesses an in-depth understanding of the post-production pipeline. In the past year, she worked on Austin PBS series like 'Taco Mafia' and 'Chasing the Tide,' served as a Production Assistant on 'Austin City Limits,' and contributed to various post-production roles on other creatively and technically demanding projects.