Friday, February 03, 2012

Jenkins User Import/Export

For work I had to whip up something quick to import and export users from Jenkins. Here's the export script:

# This should be changed to your system's jenkins root JENKINS_ROOT="/var/lib/jenkins"
JENKINS_USERS="$JENKINS_ROOT/users"
JENKINS_JOBS="$JENKINS_ROOT/jobs"
JENKINS_CONFIG="$JENKINS_ROOT/config.xml"

# This variable is used so that we can verify that a user is not found
FAIL_TEST=0

exportUserFolder() {
if [ -d "$JENKINS_USERS/$1" ]; then
cp -R "$JENKINS_USERS/$1" ./
else
FAIL_TEST=`expr $FAIL_TEST + 1`
fi
}
exportUserPermissions() {
if [ $(grep -c "^.*.*:$1<\/permission>.*" $JENKINS_CONFIG) -ne 0 ]; then
grep "^.*.*:$1<\/permission>.*" $JENKINS_CONFIG > $1.perm
else
FAIL_TEST=`expr $FAIL_TEST + 1`
fi
}

exportUsers() {
# Pre-work cleanup
if [ -d "working" ]; then
rm -R -f working
fi

if [ -f "user.tar" ]; then
rm -R -f user.tar
fi

mkdir working
cd working

# Checks if any users have been specified
if [ ! -z $1 ]; then
# Iterates over the users if they have
for u in $*; do
exportUserFolder $u
exportUserPermissions $u
# if FAIL_TEST is 2, that means both of the above functions failed to find a user
if [ "$FAIL_TEST" -ge "2" ]; then
echo "ERROR: User $u is not found!"
exit 1
fi
FAIL_TEST=0
done
else
# Otherwise we export everything!

# Copies ALL user folders
for f  in `ls "$JENKINS_USERS"`; do
exportUserFolder $f
done

# Extracts all the users for who we have global permissions
for u in `sed -n -e 's/^.*.*:\(.*\)<\/permission>.*/\1/p' $JENKINS_CONFIG  | uniq | sort`; do
exportUserPermissions $u
done
fi

# create the user.tar
# We only do this once, at the very end, so that if this fails for any reason a user.tar is not created,
# so we don't risk having a bad artifact built from the job
tar cv --file user.tar *

# Pop up to the workspace
cd ..

# move user.tar out of working into the workspace so that it can by archvied by jenkins
mv working/user.tar ./user.tar

# post-work cleanup
rm -R -f working
}

exportUsers $USERS


The only parameter needed is USERS, which is a space-separated list of users to export. If it's blank, or omitted, it exports all the users to a file called user.tar, which you should set up jenkins to archive.

Here's the import script:

INPUT_FILE="user.tar"

# This should be changed to your system's jenkins root JENKINS_ROOT="/Users/Shared/Jenkins/Home"
JENKINS_USERS="$JENKINS_ROOT/users"
JENKINS_JOBS="$JENKINS_ROOT/jobs"
JENKINS_CONFIG="$JENKINS_ROOT/config.xml"

# This variable is used so that we can verify that a user is not found
FAIL_TEST=0

importUserPermissions() {
if [ $PERMISSIONS = "true" ]; then
if [ -f "$1.perm" ]; then
# delete existing permissions
sed -i -e "/^.*.*:$1<\/permission>.*/d" $JENKINS_CONFIG
# Insert new permissions
while read line; do
sed -i -e "/^.*<\/authorizationStrategy>.*/ i\\
 $line" $JENKINS_CONFIG
done < "$1.perm"
else
FAIL_TEST=`expr $FAIL_TEST + 1`
fi
fi
}
importUserFolder() {
if [ -d "$JENKINS_USERS/$1" ] && [ "$REPLACE_CONFIG" = "false" ]; then
echo "User folder for $1 already exists."
echo "Enable REPLACE_CONFIG to overwrite existing user folders"
else
if [ -d "$1" ]; then
cp -R "./$1" "$JENKINS_USERS/"
else
FAIL_TEST=`expr $FAIL_TEST + 1`
fi
fi
}

importUsers() {
if [ -d "working" ]; then
rm -R -f working
fi
mkdir working
cd working

tar xf "../$INPUT_FILE"
# Checks if any users have been specified
if [ ! -z $1 ]; then
# Iterates over the users if they have
for u in $*; do
importUserFolder $u
importUserPermissions $u
# if FAIL_TEST is 2, that means both of the above functions failed to find a user
if [ "$FAIL_TEST" -ge "2" ]; then
echo "ERROR: User $u is not found!"
exit 1
fi
FAIL_TEST=0
done
else
# Otherwise we import everything!

# Copies ALL user folders
for f in `ls -l | awk '/^d.*/{print $(NF);}'`; do
importUserFolder $f
done
for f in `ls *.perm | sed 's/\(.*\)\..*/\1/'`; do
importUserPermissions $f
done
fi

# Pop up to the workspace
cd ..

# post-work cleanup
rm -R -f working
rm user.tar
}

importUsers $USERS



This one takes a few more parameters:

  • A file paramater writing to user.tar
  • USER, a string that specifies the user(s) to import, spaces separating multiple users. If omitted or empty, all the users in the archive will be imported.
  • PERMISSIONS, a boolean indicating wether to import permissions
  • REPLACE_CONFIG, a boolean indicating wether to overwrite existing user info
I zipped up my Jenkins jobs for easy use here

Friday, September 10, 2010

v.0.1.6 - Laurentian Living

MASGAU isn't dead. It's just in a coma. I fired up Visual Studio and tinkered with it just yesterday, honest. I've added a game data update system, fixed a bunch of bugs, and introduced a few more. There's one outstanding bug related to using DLLs that may force me to separate into 32 and 64 bit builds from now on. Ugh. We'll see. I still read everyone's contributions, and I plan on implementing every one of them. Busy life has stolen my time.

So, I moved out. At 26, maybe a bit late in the game. College is finally over. I've got a job working at the Boys & Girls Club here in Kenosha. It's not full time, but it pays the bills. I spend my days shepherding kids and trying to convince them that things aren't "bogus." They've also got me doing extra hours updating the BGC web site, but finding time for that has been more difficult than anticipated. Outside work, I've upped my commitment to my church, Missio Dei. This has made me busy in the form of helping people out with stuff, the sound team, and the church web site. I've also made more of a push to be socially active, something I really stopped working at during college. Was around people all the time, so I didn't really feel like spending my free time around more people. A lot of my friendships went by the wayside for no reason, and I'm glad I'm being given an opportunity to rebuild some of them.

I also wrote a short story last year. It's called Arrogate, and it can be found in my DeviantArt gallery. I consider it a complete story, but I'm still continuing it with another story called Abrogate. It mostly exists as a lot of notes and ideas right now, but I hope to begin actually writing it out real soon. Funnily enough I posted an unpolished version of the first chapter as "Retitle 1" (because I was going to retitle it), and even now that one chapter has by far more views than all the Arrogate chapters combined. How exactly am I supposed to take that?

I finally started and have kept with exercising. Over the last few months I've lost 38 pounds, which I monitor weekly when I go in for plasma donations. I opted to go the calorie-counting route, despite my friends who swear by Atkin's. I'm happy with the progress, and it doesn't look to be stopping any time soon.

That about gets things up to date. I'd apologize for going so long without updates, but I can really only say that so many years in a row before it becomes dishonest.

Sunday, November 15, 2009

v.0.1.5 - Startling Seven

I released 0.7 of MASGAU earlier today (yesterday). Not many new features, mostly interface and stabilizing. I decided to handle the Arkham Asylum problem by adding a save profile that just backs up the user's Game for Windows account information into it's archive. That way you can just restore the account info and the Arkham save should be ready to go. I tested it with my friend's computer earlier, and it went of without a hitch.

I've also added a fan page for MASGAU to Facebook, a link to which now exists in the right column of my blog.

Thursday, October 15, 2009

v.0.1.4 - Baffling Batman

Arkham Asylum looks like an amazing game, but it's been giving MASGAU trouble. My buddy Mikey bought the game and played it through under XP. Soon after beating it he wiped his computer and installed Windows 7. Arkham was one of the games added to MASGAU in the last update, and he used it to backup his save files. He restored the saves after re-installing the game, and MASGAU indeed restored the files to the correct location. After starting Arkham we were informed that the save was corrupt. A little Googling led me to believe that Arkham was doing something to restrict transferring saves, most likely encrypting. I finally got an opportunity to experiment on Mikey's copy of the game and came up with some interesting findings.

Inside a user's XLive folder there is a folder called content that contains per-G4W-user folders. The folder names are 16-digit hex numbers, seeming to be a concatenation of two 8-digit hex numbers. I logged in with several accounts, and found that they all had the same last 8 digits, leading me to believe these digits are a machine identifier. The first 8 digits seem to be randomly generated the first time you log in with an account to identify it on the system.

In the Arkham save folder, there is a subfolder for each of the users that have logged in and made a save. The folder name is the 8-digit account identifier mentioned in the previous paragraph, followed by 00000001. I copied a save created with my own user to Mike's user and then logged in as Mike. I was surprised to find that the game did not report the save as corrupted. This led me to believe that the copy protection was a system-specific thing. I Googled around and found a save made somewhere around the Poison Ivy battle. I copied the save into Mike's user folder and was unsurprised to see it reported as corrupted. The save had included the XLive folder of the user who made it, so I extracted it to my own XLive folder. This added an extra user, one that could see the included save game as long as it was placed in the save game folder in a folder named after the user's hex identifier. Unfortunately Mike is the only person I know who has a computer powerful enough to run the game. Once I finish repairing my system I will be able to do direct experiments with transferring saves and account info between two separate systems.

All this is of course for MASGAU's benefit. If I'm able to perfect a method, MASGAU will be able to back up the associated account info for a save along with the save. Due to the motherboard failure of my desktop, development has been very slow. I do a little work here and there at my local college, but there's only so much I can do there. As a side effect, I believe that if I am able to successfully extract account information from G4W, I might be able to make some other games that use G4W and don't protect the saves (like Fallout 3 and I think Kane & Lynch) able to automatically choose what user to restore a save to. This unfortunately may invalidate the layout of saves for those games made with older version of MASGAU, but that's why it uses 7-zip files. Due to its how it saves, Arkham will probably never be able to do this automatically. We can only hope that someday the developer will be compassionate enough to remove the copy protection from the saves.

On the upside, MASGAU has been getting some interface polishing. Almost all the windows can now be resized at will, and the main window will even remember how big you left it. There is now a status bar that indicates things that are detected, such as Steam, G4W, and maybe whether a PSP is detected. The status bar also replaces the progress splash, allowing you to see the window that much sooner. Here's a screen:
See? Progress!

Due to n employment contract snafu, I should have the money to repair my computer the Friday after next. Another release of MASGAU probably won't happen until after then, but I'm considering upping its status from Alpha to Beta. Exciting!

Thursday, September 17, 2009

v.0.1.3 - Mortal Motherboard

School has started. Work has picked up. My motherboard has died. All these things mean MASGAU development will be slower, especially for the next few weeks. There's an outstanding issue with Windows XP right now, and I was hoping to have a release ready last Sunday. That obviously didn't happen.

Tuesday, August 18, 2009

v.0.1.2 - Spreading Seagull

Well it's been a few weeks and MASGAU has already reached version 0.4 The wiki for MASGAU is finally the top result for MASGAU on Google, and a few sites have done brief write-ups on it. Here's what I've found so far:

The first I spotted

One based off the above one

This one's in Portuguese!

This one too!

Someone was kind enough to post a review on the Sourceforge page

Not really a write-up, but an independent mirror

How awesome is that? Now if only more people would submit save information...

Monday, July 27, 2009

v.0.1.1 - Mosying MASGAU

I've been spending the last month or so working on a backup program called MASGAU. It's purpose is to backup saved games from your computer, then restore them wherever you please. It uses XML definition files to automatically detect saves on your system, and can restore any backup to any detected install of the same game. I just posted version 0.2 to the site, so if you're interested, go check it out.

http://masga.sourceforge.net/

Be sure to check the Supported Games list to see if you have any games you could help out with!