I like hosting my pet projects on bitbucket.org. The free account includes both private and public projects, there's a wiki section, so far I've had zero issues with their services...
But recently I considered mirroring the projects there to other similar git cloud solutions, for backup.
After some google searching, I could just configure my local git repositories (on my PC) to "multipush" to bitbucket.org and gitlab.com (the other similar git cloud solution I ended up going for).
However, I had to replicate my "multipush" configuration accross any machine I would work at, as well as be prompted for my ssh passphrase about 4 times on each push action - and on top of that, if anyone else were to work with me collaboratively, we would all have to be extra careful in updating the cloud instances and make sure that one cloud service wouldn't get a different history than the other.
So, I decided to invest some more time with the issue and came up with what I consider to be, the ideal approach, in my case anyways:
So basically, I'm using a Cubieboard to run a Gitlab installation (as Gitlab exists both as a cloud service and as open source software that you can install and run privately), everyone pushes to that instance, and that instance alone communicates with the cloud services.
This clears any potential cloud service history mismatches as there is only one git server frontend to the developers, and ensures code replication/backup to multiple services on the cloud as well.
Note that while this setup is working, it might be worth it to consider the total volume of code generation, developers involved, etc. and replace the Cubieboard with something more robust if needed, in the long run, if that's you.
But anyways, this post is mainly about code versioning, with cloud replication, using free services and relatively cheap and low power consuming infrastructure - and Cubieboard is handling itself after 3 days non-stop usage, with a total 490MB RAM being used by the system.
That said, I'll be including my notes throughout the whole process, it took a while to get everything working, there are hacks along the way and first and foremost, the online resources I based my guide on:
(my) Cubieboard + Gitlab Tutorial
(Using Cubieboard A20 with Cubian X1 nano headless)
Table of Contents
useradd -d /home/git -m -s /bin/bash -G sudo git
passwd git
ssh git@Cubian.local -p36000
cd ~
sudo apt-get install git ruby patch ruby1.9.1-dev libmysqlclient-dev libmysqld-dev libicu-dev build-essential mysql-server redis-server libpq-dev
wget http://nodejs.org/dist/v0.10.35/node-v0.10.35.tar.gz
tar xvzpf node-v0.10.35.tar.gz
cd node-v0.10.35
./configure --without-snapshot
make -j3
sudo make install
mysql -u root -p
CREATE USER 'gitlab'@'localhost' IDENTIFIED BY 'gitlab';
SET storage_engine=INNODB;
CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;
GRANT SELECT, LOCK TABLES, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON `gitlabhq_production`.* TO 'gitlab'@'localhost' IDENTIFIED BY 'gitlab';
quit
cd ~
sudo gem install bundler --no-ri --no-rdoc
git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 7-1-stable gitlab
cd gitlab
cp config/gitlab.yml.example config/gitlab.yml
nano config/gitlab.yml
host: localhost #or 192.168.1.10, using static dhcp with the LAN router
port: 4343
# If you use non-standard ssh port you need to specify it << cubian sets sshd port to 36000, keeping it
ssh_port: 36000
chown -R git log/ tmp/
chmod -R u+rwX log/ tmp/
mkdir ~/gitlab-satellites
chmod u+rwx,g=rx,o-rwx ~/gitlab-satellites
chmod -R u+rwX tmp/pids/ tmp/sockets/ public/uploads
cp config/unicorn.rb.example config/unicorn.rb
nano config/unicorn.rb
listen "/home/git/gitlab/tmp/sockets/gitlab.socket", :backlog => 64
listen "localhost:4343", :tcp_nopush => true #or 192.168.1.10, using static dhcp with the LAN router
cp config/initializers/rack_attack.rb.example config/initializers/rack_attack.rb
git config --global user.name "GitLab"
git config --global user.email "mymail@gmail.com"
git config --global core.autocrlf input
cp config/database.yml.mysql config/database.yml
nano config/database.yml
...set DB preferences here...
nano Gemfile
#gem "therubyracer"
nano Gemfile.lock
therubyracer (0.12.0) << delete
libv8 (~> 3.16.14.0) << delete
bundle install --no-deployment --path vendor/bundle
bundle install --deployment --without development test postgres aws
confirm that the redis daemon is running (should be, at this point) > netstat -an | grep 6379
bundle exec rake gitlab:shell:install[v1.9.6] REDIS_URL=redis://localhost:6379 RAILS_ENV=production
bundle exec rake gitlab:setup RAILS_ENV=production
login.........admin@local.host
password......5iveL!fe
cp lib/support/init.d/gitlab ~/gitlab-start-stop (start as user git)
cd ~
./gitlab-start-stop start (wait for a few minutes for it to start and release the cli)
cd gitlab
bundle exec rake gitlab:env:info RAILS_ENV=production
bundle exec rake assets:precompile RAILS_ENV=production
as root: cp /home/git/gitlab/lib/support/logrotate/gitlab /etc/logrotate.d/gitlab
nano gitlab/config/gitlab.yml
-> email_from: <valid_email_address>
nano gitlab/config/environments/production.rb
-> change sendmail to smtp
config.action_mailer.delivery_method = :smtp
cp gitlab/config/initializers/smtp_settings.rb.sample gitlab/config/initializers/smtp_settings.rb
configure an email account (gmail example)
config.action_mailer.smtp_settings = {
:address => "smtp.gmail.com",
:port => 587,
:domain => 'gmail.com',
:user_name => 'account@gmail.com',
:password => 'password',
:authentication => :plain,
:enable_starttls_auto => true
}
restart gitlab
http://192.168.1.10:4343
username: root << in my case, admin@local.host didn't work, root did
password: 5iveL!fe
The user should now receive the password at his valid email address (and possibly in Junk/Spam)
On Cubian, as git user:
cd ~
nano gitlab-shell/config.yml
gitlab_url: http://192.168.1.10:4343/
restart gitlab server
Cubian sshd runs on port 36000, so on the client, edit ~/.ssh/config if you wish to facilitate things:
Host 192.168.1.10
Hostname 192.168.1.10
Port 36000
cd ~
sudo cp gitlab-start-stop /etc/init.d/
sudo -s
update-rc.d gitlab-start-stop defaults
Cubian:
git user, generate ssh keypair without passphrase
gitlab.com/bitbucket.org, add id_rsa.pub from the git user
from Cubian, test ssh access:
ssh git@gitlab.com >> Welcome to GitLab, Wickwire!
ssh git@bitbucket.org >> logged in as wickwire.
on Cubian, configure remotes for the gitlab server
for each repository, done once:
cd ~/repositories/wickwire/raspberry-pi-qrdecoder.git
cat config
[core]
repositoryformatversion = 0
filemode = true
bare = true
[remote "gitlab"]
url = git@gitlab.com:wickwire/raspberry-pi-qrdecoder.git
fetch = +refs/heads/*:refs/remotes/gitlab/*
fetch = +refs/tags/*:refs/tags/*
autopush = true
[remote "bitbucket"]
url = git@bitbucket.org:wickwire/raspberry-pi-qrdecoder.git
fetch = +refs/heads/*:refs/remotes/bitbucket/*
fetch = +refs/tags/*:refs/tags/*
autopush = true
cat hooks/post-receive
#!/bin/bash
for remote in $(git remote); do
if [ "$(git config "remote.${remote}.autopush")" = "true" ]; then
git push "$remote" --all -u
git push "$remote" --tags
fi
done
chmod a+x hooks/post-receive
gitlab reload
On client, first time synching local Gitlab and remotes:
git remote -v
git remote rm origin
git remote add origin <remote repo git url>
git push -u origin --all # pushes up the repo and its refs for the first time
(repeat for all the cloud git services)
git remote rm origin
git add origin <local gitlab server repo>
from here, add content -> git add -A -> git commit -m "msg" -> git push - and it should all work, your code should be safely pushed to all instances: the Cubian GitLab and the cloud services.
This was a little trickier as editing wiki pages using the web browser didn't seem to trigger any Cubian GitLab repository hooks, and the web hook setup for GitLab didn't seem to help either - so I ended up using a "quick and dirty hack" to make it work:
Cron job running every 5min on user git, just to push any new changes at the Cubian GitLab Wiki repo
clone the bitbucket wiki first and manually push it to gitlab Cubian and gitlab.com
git clone git@bitbucket.org:wickwire/raspberry-pi-qrdecoder.git/wiki
cd wiki
#replace the bitbucket origin with the gitlab.com origin
git remote rm origin
git remote add origin git@gitlab.com:wickwire/raspberry-pi-qrdecoder.wiki.git
#force initial push to gitlab.com
git push --force origin --all
#replace the gitlab.com origin with the gitlab cubian origin
git remote rm origin
git remote add origin ssh://git@192.168.1.10:36000/wickwire/raspberry-pi-qrdecoder.wiki.git
git push --force origin --all
On Cubian Gitlab, as git user:
~/repositories/wickwire/raspberry-pi-qrdecoder.wiki.git
cat config
[core]
repositoryformatversion = 0
filemode = true
bare = true
[remote "gitlab"]
url = git@gitlab.com:wickwire/raspberry-pi-qrdecoder.wiki.git
fetch = +refs/heads/*:refs/remotes/gitlab/*
autopush = true
[remote "bitbucket"]
url = git@bitbucket.org:wickwire/raspberry-pi-qrdecoder.git/wiki
fetch = +refs/heads/*:refs/remotes/bitbucket/*
autopush = true
cat hooks/post-receive-crontab
#!/bin/bash
WORKDIR=`dirname $0`
cd $WORKDIR
for remote in $(git remote); do
if [ "$(git config "remote.${remote}.autopush")" = "true" ]; then
git push "$remote"
fi
done
git user cron:
*/5 * * * * /home/git/repositories/wickwire/raspberry-pi-qrdecoder.wiki.git/hooks/post-receive-crontab >/dev/null 2>&1
All new entries on the wiki for this repo, using the traditional GitLab WebUI will be pushed to the cloud git repositories, every 5 minutes.
And that's it!
If you've survived this long, then you should have a working GitLab server running on your Cubieboard and while pushing code or adding/updating wiki content to it, all changes should be pushed to the cloud as well! :)
