“Test this on some non-production assets and get back to me.” This is the beginning of a miscommunication that just whipped me into a panic recently. The end result was something more like, “Please use this Bootleg Build on as many important production assets as possible. Spare no one.”
So it happened. Approximately 358 (Approximately? Okay. Maybe exactly.) data files on the cusp of finalization were rendered broken by a “test” of a batch process. Then they were immediately checked into Perforce. Individually. Over a period of thirty or so minutes. While other people checked items in and out willy nilly. Fantastic. Looks like we’ve got a problem on our hands. Instead of (or after) panicking, it was time for a Coder of Action.
Remember when I talked about Learning the Command Line? Well, this is the approach I took to the problem. If you can think of a better way, PLEASE leave a comment.
The first thing I had to do was to uncover the change lists I needed. I pulled up a command line and first got an idea of the damage done.
p4 changes -u someguy -s submitted -m 100
Okay, so it looks like he made at least 200 changes. Every change in this list took place on the day in question. Extending the -m flag (max entries to show) to 200 reveals the exact amount of change lists I needed.
p4 changes -u someguy -s submitted -m 186 > C:\GuyChanges.txt
Now I have a file containing a list of the 186 changes that were submitted by someguy. This is a good start. I’ve written some small batch files before, but I’m thinking that I need a little more power. I need to extract the number from each line and run a command on it. After a little refresher on batch operations for the command line, I give this a shot.
FOR /F “tokens=1,2,*” %X IN (C:\GuyChanges.txt) DO (p4 describe -s %Y) >> C:\GuyDescriptions.txt
This is good. I’ve now got a better description of each changelist involved. Every file involved is sitting in GuyDescriptions.txt with some chode I don’t need around it.
Change 346355 by (user) on (date) (time)
A comment.
Affected Files …
… //repository/path/here.txt#4 edit
What I need is a command line (or two) to extract the correct lines.
FOR /F “eol=C tokens=1,*” %X IN (C:\GuyDescriptions.txt) DO (echo %Y) >> C:\GuyFilesTemp.txt
This removes the first line (Change (X) by…) and grabs the second token of each other line. Effectively, I’ve removed the first line and now have a few more chode lines and the path to the repository. I process this temp file to narrow it down again. Fortunately for me, I have the luxury of knowing every comment was the same. The next command looks like this:
FOR /F “eol=I tokens=1″ %X IN (C:\GuyFilesTemp.txt) DO (echo %X) >> C:\GuyFilesTemp2.txt
Now, I know this is a bit reserved for me to do it one little command at a time, but I was being extra safe here. This effectively strips off the comments and the command that was chillin’ at the end of the file paths. One more simple command and we’ll have a pure list of files.
FOR /F “eol=f tokens=1″ %X IN (C:\GuyFilesTemp2.txt) DO (echo %X) >> C:\GuyFiles.txt
Perfect. So what do I have now? I have a list of every file affected and the revision number that was submitted. This is the revision where it broke. All I need to do now is to subtract one from that revision number and I’ll be golden. In order to do that and create a file to batch revert I created a small C# program. (I actually tried a command first, but when it faltered, I thought how easy a C# command line application would be. It ended up taking me about three or four minutes to write.)
Perforce is a lovely source version system which does not natively allow you to revert to a revision. Useful, I know. The effective work around is to sync to the head revision of a file, open it for edit, sync to the older version, resolve the file using your version, and then submit. This effectively puts the older version as the new head revision, so your file history is not lost. It does look ugly on the revision graph, though.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | using System; using System.Collections.Generic; using System.Text; using System.IO; namespace ConsoleApplication1 { class Program { static void Main(string[] args) { StreamReader file = new StreamReader("C:/GuyFiles.txt"); string contents = file.ReadToEnd(); // The files look like this: //path/to/file.txt#(broken revision) string[] files = contents.Split('#', '\n'); string[] realFiles = new string[files.Length/2]; int count = 0; foreach (string token in files) { int revision = 0; if (Int32.TryParse(token, out revision)) { // Add (revision-1) to the end of the "p4 sync //path/to/file.txt#" command string. revision -= 1; realFiles[count] += revision; count++; continue; } realFiles[count] = "p4 sync " + token + "#"; } StreamWriter newFile = new StreamWriter("C:/GuyCommands.txt"); newFile.WriteLine("p4 sync"); foreach (string command in realFiles) { // For this command we want "p4 edit //path/to/file.txt newFile.WriteLine(command.TrimEnd('#', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0').Replace("p4 sync", "p4 edit")); } foreach (string command in realFiles) { // This command will be "p4 sync //path/to/file.txt#(revision-1) newFile.WriteLine(command); } newFile.WriteLine("p4 resolve -ay"); } } } |
Okay. First, I sync to the head revision. A “p4 sync” will do. Then, I need to open all of the files for edit. For every file, I create a “p4 edit //path/to/file.txt” to open the file for edit. Then, we want to do a “p4 sync //path/to/file.txt#(revision-1)”. Then, we add a “p4 resolve -ay” to accept our changes over anyone else’s. This gives me a list of the commands to run and in the proper order. With this, I simply run another little command.
FOR /F “tokens=*” %X IN (C:\GuyCommands.txt) DO (%X) >> C:\GuySyncResults.txt
After this, I manually moved all the files into a changelist and submitted them, because Perforce is pretty great and collating everything into a list with “p4 change”. Finally, we wrap it up with a “p4 submit -c 84784″ and go on our merry way. Way to go, communication!
Update: As an aside, I am genuinely interested in hearing about how you would solve the same problem. The first comment kicked this off with a python solution, but what would you use? Do you see any ways that I could have simplified this? This was a rushed solution because fixing it immediately was the highest / only priority. Smarter solutions are always appreciated. As they say, “Work smarter, not harder.”
4 ResponsesLeave a comment ?
To save yourself the hassle of cooking fancy windows cmd shells and having to build a C# tool to parse the data you could get all that information directly from perforce to python using the '-G' flag.
import subprocess
import marshal
p = subprocess.Popen("p4 -G describe 661731")
d = marshal.load(p.stdout)
'd' comes out as a dictionary with all the information you would normally get from stdout.
Thank you! Currently learning python so this is helpful information. Wouldn't you still have to write python code to grab all of the changelist numbers? Wouldn't it basically be the same commands in the same order when spawning the p4.exe process just with the results coming back in a dictionary? How is the information stored in the dictionary? Is the key the different pieces of information ('Comment', 'AffectedFiles', etc;) and are the values just the files or do they store the action taken such as edit and add? (Note: I'm not trying to plug holes in your comment, I'm genuinely interested in other techniques that could have been used here.)
Yeah, you still need to code it in basically the same way, but with easier interaction with p4.exe.
The first command 'p4 changes' will return a bunch of dictionary entries in that form:
{'change': '661174',
'client': 'clientspecname',
'code': 'stat',
'desc': 'commentn',
'status': 'submitted',
'time': '1263930352',
'user': 'username'}
The describe command is more stupid and returns one dict with entries 'depotFile0,1,2,3,4,5,…' for all the files in the changelist. It does the same with the revision number of the file (i.e. rev0, rev1, rev2, etc …). Not as nice as getting them in lists but you can work your way around that pretty easily.
Glad you found the information useful.
Cheers
def p4command( cmd_string, input=0 ):
"""
P4 command pipe generator. P4 Command goes in. Python object
representing one result comes out until no more results.
If the command would expect a spec from the command line, use
the input – as a string – to pipe it in.
if only one object is expected use:
p4command(…).next() to get it.
"""
command = " ".join(["p4 -G", cmd_string])
(pipe_in, pipe_out) = os.popen2( command, "b")
if input:
pipe_in.write( marshal.dumps( input, 0 ) )
while True:
try:
obj = marshal.load(pipe_out)
yield obj
except EOFError:
break
pipe_out.close()
pipe_in.close()