Feedback

Tag cloud

What's your question?

By: Asked

Does a revision control system as base for a pipeline make sense?

I've seen some people build their pipeline around tools like CVS, SVN, GIT etc. I can see that it makes sense because these tools have a stable checkout and revision system. But last time I checked these systems didn't work well with binary files.

It makes no sense to me to check for differences or save file changes in a database. This makes sense for files with context like code but not for scripts from applications. So my impression was these tools do more than needed and only work with ASCII formats like ma, nk, comp etc. well.

Also in most(?) systems the files are stored in databases and not as whole file on a filesystem.

The good checkout and revision mechanism wasn't enough to justify the trouble.

Add comment viewed 635 times Latest activity 10 months ago

NN comments
or Cancel

6 answers

  • 1

julian [ Admin ]

I started doing some tests with git, and I set up a master origin for a project. But due to not getting everyone else to buy in to the idea, it fizzled out. I set it up so each shot was a repo. One shot had minutes of sourceimages (about 5 Gig) and the commit failed on that.

We had a discussion about that and other stuff on the maya_dev google group:

Here's what I posted about the few tests I made:

a few tests...

After 3 commits (of 1 100Mb file and 1 30Mb file =130Mb) the total disk usage was 154Mb

I guess it would depend on workflow whether it was useful or not. I have rotating autosaves - and I save a scene every playblast (so there's always a movie that represents the scene). As a first attempt at this. I would .gitignore the autosaves - then commit the scene file, same filename.

Plenty more to think about - what else to ignore - sysadmins always have this problem anyway and end up writing huge exclusion lists. Is it worth it? - i have no idea- maybe not. I love the idea of filenames becoming irrelevant ( like Jeff Raskin proposed.) - and branch names gaining importance ;)

Re: performance:
Branching is instant. Commits took about twice as long as a maya ascii save on 100Mb file. 50sec commit / for 23sec maya save . Not great, but it would happen in the background and you wouldn't be waiting for it.

Checkouts and switching to an existing branch were significantly faster than commits. ~5sec for 130Mb. One benefit is that the version you want to checkout will have a detailed, and instantly accessible description in the commit message - not an arcane coded filename with a number on the end. You will have a record of everything you wanted to say about each commit at the time it was commited. The commit is not just a file, but everything in the repo that worked together to make up that revision.

I think one of the biggest negative points about using git for this, is a problem of perception. Git models a directed graph - like this http://github.com/danielharan/integrity/network . Its complicated but it is what it is and its how people work and collaborate - sometimes you just have to branch off and experiment. Sometimes you want to go back to before the branch. The problem is that artists and producers often don't like looking at DAGs or even admitting that that's what they are working with. People like linear single tracks.

Git is a great version control system, but part of it's success is probably due to Github with all its cool views and reporting etc. So its worth mentioning that you can get Github on your local network.

NN comments
vfxoverflow_209
-

To be honest any source code based version control system is not really going to perform well with binaries. Git’s great for text files because it stores the deltas, but it’s not capable of doing that for binaries. This leads to larger and larger repositories and hence longer push/pull times.

or Cancel
  • 2

franssu_27 [ Editor ]

I heard of an animated feature handled like this in a French studio (using SVN if I remember well), and the checkout times went ballooning the further the production advanced.

Maybe the current breed of version control system might be better suited for VFX work, but the classic ones... not so much.

NN comments
or Cancel
  • 1

paul nendick [ Editor ]

An interesting question - and one I puzzle over routinely.

To make the question more savoury, I might point out that the new kid on the block - git - handles binary files with aplomb. And it operates exclusively on local file directories; no database required. And it does away with locking semantics - so no checkin/checkout - just branching and merging.

NN comments
or Cancel
  • 1

haggi

I think it much more useful to use an asset management system as a base and use something like svn for your coding. An asset management should be able to be extended with metadata like tasks or other database structures. Maybe you will find an asset management witch already has the required revision control tools. I think alienbrain can handle both.

NN comments
or Cancel
  • 1

wrosecrans

I currently work out of SVN as much as possible. Screenwriting is done in SVN, the Premiere Project, Nuke Scripts, and source for any internal utilities. I think it works great.

OTOH, there is a lot of stuff I wouldn't put into SVN, and I don't know if I would go that route if I was working on a big feature instead of just shorts. I've heard good things about Git, but I haven't worked with it. I think for a lot of uses outside of development, the traditional "master server" model for revision control probably makes more sense the the distributed model used by Git.

NN comments
bfloch
-

Thank you for all these answers. One thing left I am curious about: I wonder if someone has experience with GIT in a production pipeline.

or Cancel
  • 0

sergo

I've tried to use Git in our archviz company. We're working in Windows environment on 3ds max and I had some problems when moved across the git revision history in shared folders (max files was unusable). With local folders git is performing quite fast (file size is <100-200Mb) and without any problem during my test period (~2 months).

To make Git usable by the artists some custom programming/scripting is required to integrate Git commands to the software menus (or even to replace some commands completely).

I nearly started to implement some pipeline Git+3ds max, but problems with shared folders stopped me and finally I decided to implement my own custom version control software (actually just custom callback handlers for "open/save/save as" commands) without any "programming-oriented" versioning software.

NN comments
or Cancel

Question's Contributors

Related Questions