GitLab

NETWAYS Training
GitLab
Version Control System & Continuous Integration
Version: 2.5.1

GitLab

NETWAYS Training

Training Hints

  • Ask questions! Dumb questions do not exist.
  • Add your own notes! Having learnt something is reflected best with your own additions.
  • Join the conversation! Many of these exercises require real world scenarios and your feedback and questions.
  • Work together! Be part of the training team and find solutions together.

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

1 Git Introduction

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Version Control

  • Version Control System (VCS)
  • Record changes of file(s)
  • Revert changes
  • Compare changes over time
  • Who, when, what

The most simple version control system is to copy directories and add a suffix like the last date it worked. Though this might cause trouble when you're in the wrong directory. Afterall it pollutes your filesystem structure over time.

Long time ago developers invented version control systems which store the file revisions in a database.

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Centralized VCS

  • Multiple computers required file revisions
  • CVS, Subversion
  • What happens if the server is down?

CVCS

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Distributed VCS

  • Clients mirror the repository
  • Git, Mercurial, Bazaar, etc.
  • Server dies, client continues

CVCS

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

History of Git

  • Linux kernel development
    • Patches and archives
    • Proprietary software BitKeeper
  • Controversy in 2005, no more free to use
  • Kernel developers invented Git

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Design Goals for Git

  • Speed
  • Simple design
  • Non-linear development (many branches)
  • Fully distributed
  • Handle large projects in speed and size

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Git Basics

  • Learn Git, forget about other VCS systems (SVN, etc.)
  • Git interface is similar to existing VCS
  • ... but Git behaves differently

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Snapshots and Differences

  • File changes and deltas over time
    • Git Commit, take snapshot, store reference to that snapshot
    • Set of snapshots of a mini-filesystem
    • No change - link to the previous identical stored file

Basic Changes

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Work locally

  • No network latency involved as with other VCS systems
  • Local repository clone, fast operations
    • Browse the history
    • Show differences between specific branches
  • Work offline and push changes later

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Integrity

  • Everything has a check sum (SHA-1)
  • No changes possible without Git knowing about them
  • Checksums are literally used everywhere
  • Revert changes and even restore deleted files

Example:

7f0b824ba55e1fd4ffc5c461df0a0f48a94195cc

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

The three states

  • Working directory ("modified")
  • Staging area ("staged")
  • Git directory ("committed")

Basics 3 States

Modified means that you have changed the file but have not committed it to your Git database yet.

Staged means that you have marked a modified or added file in its current version to go into your next commit snapshot.

Committed means that the data is safely stored in your local database.

The working directory is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.

The staging area is a file, generally located in your Git directory, that stores information about what will go into your next commit. It is sometimes referred to as the "index", but it’s also common to refer to it as the staging area.

The Git directory is where Git stores the metadata and object database for your project. This is the most important part of Git, and it is what is copied when you clone a repository from another computer.

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Basic Git Workflow

  • Modify files in working directory
  • Stage the files which add snapshots to the staging area ("git add")
  • Commit ("git commit")
    • Takes files in staging area
    • Stores snapshot permanently in .git directory

  • If a particular version of a file is in the .git directory, it’s considered committed.

  • Staged means that the file has been modified and it was added to the staging area

  • Modified means that the file was changed since it was checked out but has not been staged yet.

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Git CLI

  • Work on the CLI
  • GUIs implement partial feature sets of the CLI tool
  • Shell sub commands
  • Bash-completion

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Git Installation

  • Available as package
  • macOS, Windows installers
  • Bash Integration
    • Completion
    • Show status and branch in the terminal

Bash Integration


~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Lab 1.1: Install Git

  • Objective:
    • Install the git package
  • Steps:
    • Use the package manager to install the git package

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Lab 1.2: Install Git Bash Integration

  • Objective:
    • Install bash completion and Git status
  • Steps:
    • Use the package manager to install the bash-completion package
    • Fetch the git-prompt.sh script from https://github.com/git/git - contrib/completion/git-prompt.sh
    • Customize your prompt in your $HOME/.bashrc file

Example:

$ vim $HOME/.bashrc
source ~/git-prompt.sh
export GIT_PS1_SHOWDIRTYSTATE=1
export PS1='[\u@\h] \W$(__git_ps1 " (%s)") \$ '

$ source $HOME/.bashrc

The training notebooks come pre-provisioned with this .bashrc configuration:

vim $HOME/.bashrc

# Enable Git branch and modifications in colored terminal
#
# Adopted from Michael Friedrich's dev environment.

# requires git bash completion
source /etc/bash_completion.d/git
# CentOS 7 specific (!)
source /usr/share/git-core/contrib/completion/git-prompt.sh

# global options
export GIT_PS1_SHOWDIRTYSTATE=1
#export GIT_PS1_SHOWSTASHSTATE='y'
#export GIT_PS1_SHOWUNTRACKEDFILES='y'
#export GIT_PS1_DESCRIBE_STYLE='contains'
export GIT_PS1_SHOWUPSTREAM='git'

function myPrompt() {
        local GREEN_BOLD="\[~~~CONTENT~~~33[01;32m\]"
        local RED_BOLD="\[~~~CONTENT~~~33[01;31m\]"
        local BLUE_BOLD="\[~~~CONTENT~~~33[01;34m\]"
        local GREEND="\[~~~CONTENT~~~33[02;32m\]"
        local REDD="\[~~~CONTENT~~~33[02;32m\]"
        local DEFAULT="\[~~~CONTENT~~~33[00m\]"

        # \h ... hostname
        # \w ... workdir
        # \$? ... RC
        # \u ... user
        # \$(formattedGitBranch) ... git branch if available

        local USER_BOLD=$GREEN_BOLD
        local USERD=$GREEND
        local USERAT="\u@\h"
        local USERHASH="\$"

        if [ `/usr/bin/whoami` = 'root' ]
        then
                USER_BOLD=$RED_BOLD
                USERD=$REDD
                USERAT="\h"
                USERHASH="#"
        fi

        PS1="$USER_BOLD$USERAT$DEFAULT $BLUE_BOLD\w$DEFAULT"
        PS1=$PS1"$DEFAULT$GREEND\$(__git_ps1)$BLUE_BOLD $USERHASH $DEFAULT"

        # window title
        case "$TERM" in
                xterm*|rxvt*)
                        PS1="\[\e]0;\h: \w (\u)\a\]$PS1"
                        ;;
        esac
}

set git colored prompt based on uid

myPrompt

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

More Git Shell Integrations

Powerline Shell

Powerline Shell Integration

Windows Powershell: git-posh

Powershell Integration


Powerline Shell Integration

The Powerline shell integration can be found here: https://github.com/b-ryan/powerline-shell

# yum -y install epel-release
# yum -y install python-pip
# pip install powerline-shell

@@@ Sh
$ vim $HOME/.bashrc

function _update_ps1() {
    PS1=$(powerline-shell $?)
}

if [[ $TERM != linux && ! $PROMPT_COMMAND =~ _update_ps1 ]]; then
    PROMPT_COMMAND="_update_ps1; $PROMPT_COMMAND"
fi

Powerline needs an additional font which can be downloaded from here: https://github.com/powerline/fonts/blob/master/Meslo%20Slashed/Meslo%20LG%20M%20Regular%20for%20Powerline.ttf

Choose Raw and install it into your system.

Modify your terminal profile settings and choose Meslo LG as font.

Windows Powershell Integration

The Windows Powershell integration requires NuGet. Open a new Powershell prompt and enter:

Install-Module git-posh
Add-PoshGitToProfile

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Git GUI Clients

In this training, we are working with the CLI. For your work environment, here are a few recommendations for GUI clients:

  • Sourcetree (Windows, macOS)
  • GitKraken (Windows, Linux, macOS)
  • TortoiseGit (Windows)
  • GitExtensions (Windows)
  • GitHub Desktop (Windows, macOS)

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Git GUI Clients: Sourcetree

GUI Client: Sourcetree

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Git GUI Clients: Gitkraken

GUI Client: Gitkraken

~~~SECTION:MAJOR~~~ Git Introduction

NETWAYS Training

Git GUI Clients: GitHub Desktop

GUI Client: GitHub Desktop

~~~SECTION:MAJOR~~~ Git Configuration

NETWAYS Training

2 Git Configuration

~~~SECTION:MAJOR~~~ Git Configuration

NETWAYS Training

Configuration Overview

  • Global configuration for the user
    • $HOME/.gitconfig
  • Local to the repository
    • training/.git/config

~~~SECTION:MAJOR~~~ Git Configuration

NETWAYS Training

Configuration

  • CLI command support
  • Edit $HOME/.gitconfig directly

Example:

$ git config --global user.email michael.friedrich@netways.de

$ cat $HOME/.gitconfig
[user]
email = michael.friedrich@netways.de

More information can be found in the documentation at https://git-scm.com/book/tr/v2/Customizing-Git-Git-Configuration

~~~SECTION:MAJOR~~~ Git Configuration

NETWAYS Training

Configuration Sections

  • Commit author (user)
  • Aliases (alias)
  • Colors for diff and verbose commit (color)
  • Core functionality (core)

Example:

Git Config


~~~SECTION:MAJOR~~~ Git Configuration

NETWAYS Training

Lab 2.1: Configure your username and email address

  • Objective:
    • Configure your username and email address using Git CLI commands
  • Steps:
    • Use git config --global user.name "Your Name"
    • Use git config --global user.email "name@domain.com"
  • Next steps:
    • Verify the changes with git config --global --list

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

3 Git Basics

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Git Command Overview

  • Start a working area (clone, init)
  • Work on current changes (add, reset)
  • Examine the history and state (status, log)
  • Grow, mark and tweak the history (branch, checkout, commit, merge, rebase)
  • Collaborate (fetch, pull, push)

Example from Git CLI command:

start a working area (see also: git help tutorial)
   clone      Clone a repository into a new directory
   init       Create an empty Git repository or reinitialize an existing one

work on the current change (see also: git help everyday)
   add        Add file contents to the index
   mv         Move or rename a file, a directory, or a symlink
   reset      Reset current HEAD to the specified state
   rm         Remove files from the working tree and from the index

examine the history and state (see also: git help revisions)
   bisect     Use binary search to find the commit that introduced a bug
   grep       Print lines matching a pattern
   log        Show commit logs
   show       Show various types of objects
   status     Show the working tree status

grow, mark and tweak your common history
   branch     List, create, or delete branches
   checkout   Switch branches or restore working tree files
   commit     Record changes to the repository
   diff       Show changes between commits, commit and working tree, etc
   merge      Join two or more development histories together
   rebase     Forward-port local commits to the updated upstream head
   tag        Create, list, delete or verify a tag object signed with GPG

collaborate (see also: git help workflows)
   fetch      Download objects and refs from another repository
   pull       Fetch from and integrate with another repository or a local branch
   push       Update remote refs along with associated objects

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Start a project

  • git clone
    • Clone a copy of existing remote repository.
    • Can be newly created in GitLab/GitHub and empty.
  • git init
    • Initialize a local empty Git repository.

git clone clones a copy of an existing remote Git repository.

git init initializes an empty directory as local Git repository.

When you clone a repository, Git automatically adds a shortcut called origin that points back to the "parent" repository, under the assumption that you'll want to interact with it further on down the road. This is called an origin.

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Lab 3.1: Clone an existing Git repository

  • Objective:
    • Clone an existing Git repository
  • Steps:

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Lab 3.2: Initialize a local Git repository

  • Objective:
    • Initialize git repository
  • Steps:
    • Create a new directory called training in your home directory
    • Change into it
    • Run git init

We will be working inside the training directory throughout the training unless noted otherwise.


~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Add current changes

  • git add
    • Add new files to Git (staged)
    • Add modified file(s) from the working directory into the staging index.
    • -A adds all files. Question: Where is this applicable?
  • git mv
    • Rename file(s) tracked by Git.

git add will add the file(s) and their content to the staging index waiting for the commit. This applies to both, new files and also modified files.

If you're using -A to add all files, you need to ensure that unwanted files are not added. Learn more about git reset in the next slide to selectively unstage added changes.

git mv renames an existing file tracking the change for the commit. If you manually move the file, you will need to rm and add it again.

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Lab 3.3: Add a new README.md file

  • Objective:
    • Add a new README.md file
  • Steps:
    • Change into $HOME/training
    • Create README.md and add # GitLab Training Notes as first line
    • Use git add to add README.md to the current change index
  • Next steps:
    • Verify the change with git status

Best practice is to have a README.md file written in Markdown in every project. This gets rendered by GitHub/GitLab in readable HTML.

During this training we will learn many new things. Keep notes in the README.md file.


~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Remove changes

  • git reset
    • Reset files added to the staging index.
    • Hint: This comes in handy with git add -A before.
  • git rm
    • Remove the file(s) from working tree and Git repository.
    • Note that file(s) will be visible in Git history, and can be restored from it.
  • git rm --cached
    • Alternative suggested by Git CLI for git reset

git reset resets files added to the staging index. You can also use it to reset commits from the history.

This also is helpful when you need to add 95 out of 100 changes. First, use git add -A and then selectively unstage the unwanted 5 changes.

git rm removes the file from the working tree and also from the git index.

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Lab 3.4: Reset File from Staging Index

  • Objective:
    • Reset file from staging index
  • Steps:
    • Change into $HOME/training
    • Remove the previously added README.md file from the staging index with git reset README.md
    • Verify it with git status and explain what happened.
    • Re-add the README.md and examine again with git status.

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Examine the current state

  • git status
    • Show current working tree status.
    • Modified files
    • Modified and added to staging for commit
    • Untracked files

Later we will learn how to compare specific commits and branches too.


git status shows the current working tree status. Untracked files and changes (not) staged for commit.

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Lab 3.5: Examine current changes

  • Objective:
    • Examine current changes
  • Steps:
    • Change into $HOME/training
    • Edit README.md and add notes
    • Use git status to see unstaged changes
    • Add the changed files to the staging area
    • Use git status again

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Examine the current state: Diff

  • git diff
    • Compare changes between modified working tree and latest commit.
    • Output is a unified diff similar to diff -ur file1 file2.

Later we will learn how to compare specific commits and branches too.


git diff shows changes between the current working tree and the last commit. You can also compare specific commits.

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Lab 3.6: Use Git Diff

  • Objective:
    • Use git diff
  • Steps:
    • Change into $HOME/training
    • Edit README.md
    • Use git diff to compare unstaged changes
    • Add the changed file to the staging area
    • Use git diff again
    • Explain what git diff --staged does

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Exclude files with .gitignore

  • Build directories from source code compilation (e.g. debug, release)
  • Files generated at runtime (e.g. test results or stats)
  • User specific IDE settings
  • Local Vagrant boxes and other temporary files

You can use wildcard pattern matches for files. If you'll end the string with a '/' git will only ignore directories instead of both directories and files matching the pattern.

If you want to ignore a file globally in your repository you can preceed the path with **/ which means any directory.

Example for ignoring a debug file anywhere in your repository:

**/debug

If you prefer to keep .gitignore files in specific directories and not a global file, this will also work.

~~~SECTION:MAJOR~~~ Git Basics

NETWAYS Training

Lab 3.7: Add .gitignore file and exclude files/directories

  • Objective:
    • Add .gitignore file and exclude files/directories
  • Steps:
    • Change into $HOME/training
    • Create a file generated.tmp
    • Create a directory debug with the file .timestamp
    • Examine the state with git status
    • Exclude them in a .gitignore file
    • Examine the state with git status

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

4 Git Commits

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Git Commits

  • View as history log on production changes
  • Keep the subject short, simple and explaining
  • Make it easier for others understanding the changes
  • Add detailed explanations on larger changes
  • Add references to existing commits and/or ticket ids for further details

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Work on Git History

  • git commit
    • Selected changes from the staging index
    • All changes (-a) not necessarily added to the staging index
  • Verbose mode shows changes compared to latest commit (-v)
  • Uses the configured editor (vim, nano, etc.)
    • -m allows for a short commit message without editor view

Commits use the user.name and user.email settings to qualify you as the author of this change. In addition to that the date and time is stored.


git commit collects and records all changes stages for commit. It uses the configured user name and email address as commit author. This command opens the configured editor requiring you to add a commit message.

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Lab 4.1: Commit Changes

  • Objective:
    • Modify files and commit your changes
  • Steps:
    • Change into $HOME/training
    • Modify the README.md file and add more docs
    • Add the change to the changing index
    • Commit the change to your Git history with git commit -v README.md
  • Next steps:
    • Use git log to verify the history

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Good Commits

  • Selectively add changes for commit (do not commit everything)
    • git add for all changes in a file
    • git add -p for interactive selection of changes in a file
  • git commit <file1> <file2>
  • Enable the verbose mode -v to show the differences below the editor
    • Write a short summary based on the visible changes

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Commit Message Overview

  • Pick a short telling subject (max. 80-120 characters)
  • Add a new line
  • Add a body text explaining the issue (max. 80-120 characters in a line)
  • Optional: Add external reference markers, e.g. for ticket systems

Example:

A short subject for the commit line
<newline>
Some body text explaining the issue.
80-120 characters max width.
<newline>
refs #<ticketid>

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Examine the Git history

  • git log
    • Show commit history of the current branch.
    • Supports -10 notation to show a limited number of commits.

git log shows the commit history of the current branch.

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Examine the Git history

  • git show
    • Print commit details.
    • If the commit id is omitted, the last commit is printed.
    • Supports -10 notation to show a limited number of commits.
  • git diff
    • Show changes between working tree and last commit.
    • Supports source and target parameters.
    • Can be used to compare 2 commit ids, branches, etc.

git show will print the last commit details. If you want to print a specific commit id, add it afterwards.

git diff shows changes between the current working tree and the last commit. You can also compare specific commits.

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Lab 4.2: Examine the Commit History

  • Objective:
    • Examine the commit history
  • Steps:
    • Change into $HOME/training
    • Add and commit remaining changes e.g. .gitignore
    • Use git log to print the current history
    • Use git show to show specific commits (defaults to the latest)
    • Use git diff to compare changes between specific revisions

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Advanced history with tig

  • tig helps visualize history and branches
  • Inspect specific commits while scrolling

Tig

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Lab 4.3: Learn more about tig

  • Objective:
    • Install and use tig
  • Steps:
    • Install the tig package
    • Run tig in $HOME/training
    • Clone a different repository and run tig there e.g. $HOME/icinga2
  • Next steps:
    • Select a line and press Enter
    • q quits the detail view and the application

~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Amend changes to commits

  • Change the commit message, e.g. typos or missing changes broke the build
  • Amend changes from staging
  • Helps if new files were added but not committed
  • git commit --amend changes the latest commit
    • Amending commits in Git history is possible, explained later with git rebase.

If you amend changes to a specific commit, a new unique commit id is generated. This changes the Git history and we will learn later how to resolve possible problems in collaboration with others.


~~~SECTION:MAJOR~~~ Git Commits

NETWAYS Training

Lab 4.4: Amend changes to commits

  • Objective:
    • Use git amend
  • Steps:
    • Change into $HOME/training
    • Modify README.md and add docs about amend
    • Add README.md to the staging index and commit the change
    • Edit README.md again and add it to staging
    • Use git commit --amend README.md and explain what happens
  • Bonus:
    • Adopt the commit message using git commit --amend

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

5 Git Branching

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Working with Git branches

A git branch creates a new history line starting from the current git commit.

Branches are useful to develop features/fixes in their isolated environment.

  • Master branch
  • Develop a new feature in a dedicated branch
  • Put fixes into the master branch (production)
  • Continue to work on the feature

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Git Branch CLI commands

  • git branch
    • Create new branch without leaving the current one.
    • Delete branches
    • List branches
  • git checkout
    • Switch between branches

git branch allows you to list, create and delete branches.

git checkout will switch between branches.

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Lab 5.1: Show the current branch

  • Objective:
    • Show the current branch
  • Steps:
    • Change into $HOME/training
    • Use git branch to highlight the current branch

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Lab 5.2: Create and checkout a new branch

  • Objective:
    • Create and checkout a new branch
  • Steps:
    • Change into $HOME/training
    • Create a new branch feature/docs based off master with git branch feature/docs master
    • List the branches with git branch
    • Checkout the new branch with git checkout feature/docs
  • Bonus:
    • Explain git checkout -b feature/docs2

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Lab 5.3: Delete the branch

  • Objective:
    • Delete the previously created branch
  • Steps:
    • Change into $HOME/training
    • Switch to the master branch
    • Use git branch -d to delete the selected branch
  • Bonus:
    • Try to delete the branch you are currently on

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

HEAD and "smart pointers"

A git commit is unique and identified by its SHA1 checksum.

HEAD is a pointer to the latest commit in the current branch.

HEAD^ identifies the second latest commit.

HEAD~9 points to the tenth latest commit. Counting starts at zero.

This can be used for git show. More advanced techniques will be discussed later.

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Lab 5.4: Show the second commit

  • Objective:
    • Use HEAD and only show the second latest commit.
  • Steps:
    • Change into $HOME/training
    • Combine git show with HEAD^ or HEAD~1

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Branches and "smart pointers"

feature/docs as branch name also points to the latest commit.

You don't need to change branches to

  • show different branch histories
  • show specific commits where you don't know the commit id
  • compare branches and committed code

Example:

$ git show feature/docs
commit b825ff86e4022a8fbcf52cb5a1d9a1984bd2a310 (feature/docs)

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Lab 5.5: Show history of different branch

  • Objective:
    • Use git log from the master branch on another branch
  • Steps:
    • Create a new branch aside from master, if not existing: git checkout -b feature/docs
    • Switch to the master branch
    • Use git log feature/docs
  • Bonus:
    • Modify and commit changes
    • Diff current HEAD against feature/docs branch

~~~SECTION:MAJOR~~~ Git Branching

NETWAYS Training

Reset Git Commits

  • git reset
    • Remove the current commit(s).
    • --soft adds changes to the staging index.
    • --hard drops them.
  • Requires a new commit base specified with HEAD or any other pointer.
    • HEAD~3 points to the 4th commit back in the history, resetting the latest 3 commits

Try it out with the trainer.

$ git reset --soft HEAD^
$ git reset --hard HEAD^

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

6 Git Server

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Introduction

  • Central storage for repositories
  • Collaboration between teams
  • User based access control
  • Trigger events (e.g. for CI)

There is a variety of Git server tools, web interfaces and addons out there.

  • GitLab
  • gitosis
  • gitolite

In case you don't want to host your own Git server, there are open source and enterprise hosting options available.

NETWAYS also provides GitLab hosting services:

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Git Server Overview

  • Git server daemon
  • Web interfaces
  • Entire collaboration suites
    • GitHub
    • GitLab
    • Bitbucket

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

GitLab

GitLab

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

GitHub

GitHub

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Git Server Protocol

  • Read/write access via SSH
    • git@github.com:username/repo.git
  • HTTPS protocol (write access via oauth tokens)
    • https://my-gitlab.nws.netways.de/username/repo.git
  • Git protocol
    • git://domain.com/repo.git
  • Local protocol
    • file:///opt/git/repo.git

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

GitLab Introduction

GitLab is available as self-hosted or cloud based repository management system.

  • Git repositories
  • User and group management and fine granular permissions
  • Issue tracking and project management (dashboards, etc.)
  • Merge, review, report
  • Continuous integration/deployment (CI/CD)

Hosted: https://www.netways.de/managed_hosting/gitlab_ce_hosting/

NWS: https://nws.netways.de/products/gitlab-ce


~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

GitLab Editions

  • Community Edition (CE), part of this training
  • Enterprise Edition (EE), additional features
    • Multiple LDAP servers
    • LDAP group import
    • Merge Request Review Approvals
    • Git Hooks (not web hooks)
    • Search backend with Elasticsearch

Overview: https://about.gitlab.com/features/ Comparison: https://about.gitlab.com/images/feature_page/gitlab-features.pdf

The source code is publicly available for both editions. You'll need a valid license for running EE in production.

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

GitLab Components

  • Ruby on Rails application (unicorn, sidekiq)
  • Nginx webserver
  • PostgreSQL database backend
  • Redis cache
  • NodeJS for Javascript rendering
  • Golang for background daemons

It is recommended to use the Omnibus installation package or use a managed cloud hosting service.


Omnibus packages: https://about.gitlab.com/installation/?version=ce

More details on the manual installation instructions can be found in the official documentation: https://docs.gitlab.com/ce/install/installation.html

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Git Server Installation

This training focuses on a pre-installed GitLab instance.

Each user gets access to NWS - https://nws.netways.de and a GitLab CE app.

GitLab supports working with

  • SSH
  • HTTPS

Manual installation instructions are provided in the handout.


~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Lab 6.1: Create GitLab app in NWS

  • Objective:
    • Create a new GitLab app in NWS
  • Steps:
    • Navigate to https://nws.netways.de and register a trial account if not existing
    • Choose Apps > GitLab CE > Basic
    • Deploy the app
    • Choose Access and Live View and set a secure password for the root user.
    • Login

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Connect Local Repository to Remote Server

  • Local standalone repository
  • Connect to remote server
  • Clone, Pull, Fetch, Push via SSH/HTTPS

You can also start fresh without any local repository and clone that from remote.

For training purposes we've started to work offline in $HOME/training. Now we want to publish the local commits to a newly created Git repository in GitLab.


~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Requirements

  • SSH or HTTPS auth
    • NWS apps come pre-defined with HTTPS clone/fetch only.
  • New GitLab repository for this user
  • Configure local repository for the remote server

SSH Keys

Generate a new SSH key pair on your client.

ssh-keygen

Copy the public key into your GitLab settings.

cat $HOME/.ssh/id_rsa.pub

User > Settings > SSH Keys

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Lab 6.2: Create GitLab Project

  • Objective:
    • Create a new GitLab project for the current user
  • Steps:
    • Click the + icon next to the search field
    • Choose New Project
    • Add the name training
    • Leave it as Private
    • Create the project
  • Note:
    • Learn about the project view and the HTTPS clone URL

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Lab 6.3: Configure Client Credentials Helper

  • Objective:
    • Configure client credentials helper
  • Steps:
    • Install the helper with yum -y install git-gnome-keyring
    • Run git config --global credential.helper /usr/libexec/git-core/git-credential-gnome-keyring
    • Verify this setting in ~/.gitconfig

Navigate into NWS > GitLab App View > FAQ > Why do we recommend HTTPS over SSH? and select git credential helpers.


~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Lab 6.4: Add the repository as remote origin

  • Objective:
    • Add the GitLab project as remote origin
  • Steps
    • Open the project in GitLab and extract the HTTPS clone URL
    • Navigate into your local repository in $HOME/training
    • Use git remote add origin <remoteurl>
    • Push your local branch with --set-upstream (short: -u)
  • Bonus
    • Set default push method to simple
    • Explain what git push -u origin --all does suggested by GitLab

~~~SECTION:MAJOR~~~ Git Server

NETWAYS Training

Lab 6.5: Explore Project History

  • Objective:
    • Learn more about GitLab and the project's history
    • Compare the local history to the remote project's history
  • Steps:
    • Click on History in the project view and examine the Git commits
    • Run git log or tig on your shell and compare them to GitLab
  • Bonus:
    • Use Repository > Graph in GitLab

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

7 Collaboration with others

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Collaboration

  • Work locally
  • Push to remote repository
  • Share your work with others
  • Review, discuss, collaborate
  • Change, adopt, release

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Collaboration: Put History

  • git push
    • Update remote references and push local history to remote repository.
    • This pushes source code changes and commits.
    • Halts if the remote history diverged from your local history.
  • git remote
    • Configure/list remote repository URLs (default origin).
  • git branch -r
    • List remote branches, prefixed with the remote name, e.g. origin/master.

git push updates remote references and pushes your local commit history to the remote repository.

git remote allows you to configure and list the remote repository. By default this is called origin.

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Lab 7.1: Learn more about git push

  • Objective:
    • Learn more about git push
  • Steps:
    • Change into $HOME/training
    • Edit README.md and add a note on git push
    • Add and commit the changes
    • Push the changes
  • Bonus:
    • List all remote branches with git branch -r

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Collaboration: Branch Tracking

  • Local branches track remote branches
  • Use same names for local and remote branches
    • Enables "simple" push with git push in the current checked out branch
  • git branch -a lists all branches
  • Use git branch -vv to list tracking

Tracking

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Collaboration: Get History

  • git fetch
    • Update the remote branch reference pointers to the latest commit and cache it locally.
    • Does not pull in any remote commit history.
  • git pull
    • Fetch and update the local history from remote repository (implicit fetch).
    • This pulls in source code changes and commits.

git fetch downloads objects and references from another remote repository.

git pull invokes a fetch and updates the local history with commits from the remote repository.

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Lab 7.2: Learn more about git fetch and git pull

  • Objective:
    • Learn more about git fetch and git pull
  • Steps:
    • Go to your project repository in GitLab
    • Edit the README.md in your browser and commit the change to master
    • Run git fetch and explain git diff master origin/master
    • Run git pull
    • Explain the difference

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Git Tags

  • Tag specific points in history
  • Add, list, delete
  • Push tags to remote repository
  • Checkout branches based on tags
  • Release software versions based on tags (e.g. v2.9.0)
  • Tags can/should follow milestone versions in ticket systems

Example for checking out a tag into a new branch:

$ git checkout -b version01 v0.1
$ git branch

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Lab 7.3: Add Git Tag

  • Objective:
    • Add git tag
  • Steps:
    • Use git tag and add the v0.1 tag
    • Verify the added tag with git tag -l
    • Push tags to remote origin with git push --tags
    • Open GitLab and navigate into Repository > Tags
  • Bonus:
    • Add a tag description with git tag -m "Release v0.1" v0.1

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Advanced Git Commands: Stash

  • git stash
    • Put current changes on a temporary stack.
    • Useful when changing branches (diff would not apply)
    • Use with care, pop changes immediately after changing back
    • Only local, not stored in the central repository

Example:

$ git stash
Saved working directory and index state WIP on master: 4b4f6c2 <msg>

$ git stash pop
Dropped refs/stash@{0} (43d879b99aca12b6175c5362339b177af22589a9)

git stash allows you put your current changes on a temporary stack (stash). This comes in handy when you want to change branches with a different history where your uncommitted changes will not apply. Use git stash pop to fetch the changes again. You can stash multiple uncommitted stages, git stash list will list them.

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Lab 7.4: Learn more about git stash

  • Objective:
    • Learn more about git stash
  • Steps:
    • Change into $HOME/training
    • Edit README.md
    • Examine the status with git status
    • Stash your current changes to the working directory
    • Run git status again
    • Examine the stash with git stash list and git stash show -p
    • Fetch the previously stashed changes with git stash pop

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Advanced Git Commands: Cherry-Pick

  • git cherry-pick
    • Collect specific commit into your working tree.
    • Applies the contained patch
    • When the base commit differs, checksum changes = new commit id
    • Use git cherry-pick -x <sha1> to add source comment

git cherry-pick collects a specific commit into your working tree.

~~~SECTION:MAJOR~~~ Collaboration with others

NETWAYS Training

Lab 7.5: Learn more about git cherry-pick

  • Objective:
    • Learn more about git cherry-pick
  • Steps:
    • Create and checkout the feature/docs-hotfix branch
    • Edit README.md and commit the change
    • Use git log -1 to examine the Git commit
    • Checkout the master branch
    • Use git cherry-pick -x <id>
    • Verify the commit with git show

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

8 Git Workflows

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Git Workflows

  • Single user environments
  • Multiple users working together
  • Branching models
  • Merge and pull request workflow integration

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Centralized Workflow

  • Developers clone the central repository
  • Work in local copies
  • New commits are stored locally
  • Importing the remote repository's changes is optional

Centralized Workflow

Compared to other VCS systems, Git provides the advantage of also storing a local copy of the repository and branches.

That way a developer can work isolated on changes on its own while other developers can do the same. These isolated environments ensure that each developers works independantly from other changes in the project.

In addition to that the Git branching model provides a fail-safe mechanism for integrating and sharing code changes between repositories.

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Centralized Workflow - Publish Changes

  • Developers push their local master branch
  • Stored in central repository
  • Adds all local commits that are not in the central master branch

Centralized Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Centralized Workflow - Managing Conflicts

  • Central repository's commit history is important
  • If local commit history diverges, pushing changes is denied

Centralized Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.1: Collaborate in a central repository

  • Objective:
    • Create conflicting history tree
  • Steps:
    • Open the GitLab project training
    • Edit README.md, add This change is from my colleague..
    • Stage & commit the change to master
  • Local steps:
    • Change into training directory
    • Edit README.md, add This is my local change..
    • Commit and try to push, explain the error message

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Centralized Workflow - Managing Conflicts: Rebase History

  • Fetch the remote history
  • Rebase local changes on top of it
  • Linear history

Centralized Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.2: Resolve conflicts in a central repository

  • Objective:
    • Rebase your local history with the remote repository
  • Steps:
    • Fetch remote with git fetch
    • Compare changes with git diff origin/master
    • Rebase with git rebase origin/master
    • Resolve possible merge conflicts, add them
    • Continue with git rebase --continue, push rebased history

Hint: >>> marks conflicts and show the differences on merge/rebase.


Instead of git fetch and git rebase you can also use the git pull command with the additional --rebase flag. This helps if you are familiar with SVN and svn update.

If you forget the --rebase flag it will still work but generate merge commits. This will merge your commits in historical order but not rebase them on top of the existing history.

For using a centralized workflow it is better to use rebase instead of generating a merge commit.

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Feature Branch Workflow

  • Feature development happens in named branches
  • Does not interfere with the main codebase
  • Master branch does not contain broken code
  • Feature branches can be rebased against stable master branch on demand

Feature Branch Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Feature Branch Workflow - How it works

  • Create a new branch for each feature
  • Descriptive branch names, e.g. feature/docs-workflows
  • Changes in a feature branch similar to centralized workflow
  • Push feature branches to central repository for collaboration with other developers

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.3: Use Feature Branches

  • Objective:
    • Create a new feature branch feature/docs-workflows
  • Steps:
    • Change into $HOME/training
    • Use git checkout -b feature/docs-workflows to create a new feature branch based on the master
    • Add and commit changes
    • Push the branch to your central repository

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.4: Merge Feature Branches

  • Objective:

    • Update master branch and merge feature branch feature/docs-workflows
  • Steps:

    • Checkout the feature branch feature/docs-workflows
    • Edit README.md, add and commit the changes
    • Diff the feature branch to the current master with git diff master
    • Checkout the master branch, merge the feature branch as non-fast-forward with --no-ff
    • Show the history tree with tig or inside GitLab and explain why the forced merge commit with --no-ff is important

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Gitflow Workflow

  • Strict branching model for project releases
  • Based on Feature Branch Workflow
  • Assigns roles to different branches
  • Defines interaction between branches for releases
    • Prepare
    • Maintain
    • Record

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Gitflow Workflow - Historical Branches

  • Master branch for release history (including version tags)
  • Master branch is always stable and in production
  • Develop branch for feature integration

Gitflow Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Gitflow Workflow - Feature Branches

  • New features in their own branches
  • Feature branches use develop as their parent branch
  • Once completed, merged back to develop

Gitflow Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Gitflow Workflow - Release Branches

  • Enough features in develop: release branch based on develop
  • Ready to ship: merged to master and tagged with version

Gitflow Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Gitflow Workflow - Maintenance Branches

  • Fixes based on master
  • Merged to master, tagged and merged to develop

Gitflow Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Forking Workflow with GitHub

  • Fork aka "create a copy" of the server-side repository
  • Commit and push into own repository copy
  • Create a Pull Request for original repo, CI runs automated tests
  • Developers/Maintainers review and merge PR, CI triggers deployment task

GitHub Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Forking Workflow - Multiple Remote Repositories

Example:

$ git remote -v
origin  https://github.com:dnsmichi/icinga2.git (fetch)
origin  https://github.com:dnsmichi/icinga2.git (push)
upstream    https://github.com/icinga/icinga2.git (fetch)
upstream    https://github.com/icinga/icinga2.git (push)

Forking Workflow - Keep in Sync

  • Pull changes from upstream HEAD to own repository master branch to sync development
  • Remote HEAD is the default branch for this repository, usually master
  • Remember, HEAD is just a smart pointer. You can use a branch name too.

Example:

$ git checkout master
$ git fetch upstream
$ git pull upstream HEAD
$ git push origin master

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

GitLab Workflow

  • Manage branch access with roles and permissions
  • Create Merge Requests from branches, CI runs tests/builds automatically
  • Review and inline code comments
  • Project maintainer merges requests and manages issues/milestones/releases

Gitlab Workflow

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.5: Create Milestone and first Issue

  • Objective

    • Create Milestone v0.1
    • Create Issue Update documentation
  • Steps:

    • Navigate into Issues > Milestones
    • Select New Milestone and use v0.1 as title
    • Navigate to Issues and select New issue
    • Use Update documentation as title, add a description
    • Assign the v0.1 milestone

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.6: Create Merge Request

  • Objective:

    • Create merge request from feature branch
  • Steps:

    • Create/checkout the branch feature/docs-merge-request
    • Edit README.md, add, commit and push the changes
    • Open the proposed GitLab URL in your browser
    • Fill in the merge request and add fixes #1 as description
    • Merge the MR and tick delect source branch
    • Analyse the history in GitLab/tig and open issue #1
  • Bonus:

    • Run git fetch --prune and git branch -d feature/docs-merge-request

For future branch cleanups, you'll need to compare the deleted remote branch references from git fetch --prune with the local branch names, review them and then delete them.

$ git branch -vv | grep 'origin/.*: gone]' | awk '{print $1}'
$ git branch -vv | grep 'origin/.*: gone]' | awk '{print $1}' | xargs git branch -d

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Workflow Examples: Icinga 2

  • Feature branch workflow
  • master as development branch
  • Major releases from master, support/x.y for bugfix releases

Moving to GitHub, we extended this to:

  • (WIP) Pull Requests, local and community forks
  • Travis CI (Linux) & Appveyor (Windows) automated tests
  • Developers review and approve PRs, maintainer merges
  • Maintainer prepares and publishes a release

https://github.com/Icinga/icinga2/blob/master/CONTRIBUTING.md https://github.com/Icinga/icinga2/blob/master/RELEASE.md


~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Workflow Examples: Icinga Exchange

  • Issue management in GitLab with milestones and boards
  • Developers create Merge Requests from local branches
  • CI jobs run unit and Mock tests (GitHub oauth, etc.)
  • Maintainer reviews MRs and merges
  • Maintainer creates a new release
  • Manual deployment via Puppet and release tags

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

More Hints: Rebase and Squash

The git rebase command can also be used to perform certain actions on a specific commit history.

If you are contributing to open source projects developers might ask you to either rebase your history or even squash all commits into one commit.

commit1  => commit
commit2
commit3

Ask the trainer to draw an image for better illustration and discussion.

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.7: Rebase and squash commits

  • Objective:
    • Rebase and squash commits
  • Steps:
    • Add 3 commits to the master branch and push them.
    • Use git rebase -i HEAD~3 to start the interactive mode. HEAD~3 takes the last 3 commits compared to current HEAD.
    • Use pick for the top commit
    • Replace pick with squash for the other commits
    • Save and edit the final commit message
    • Use git log to verify the history

git rebase can also be used to edit the commits in your history. This is helpful if you want to amend changes to the test commit. Or if they are missing issue references, or just contain wrong information.

This is a common scenario for code reviews before merging the actual history into the development branches.

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.8: Force Push and Protected Branches

  • Objective:
    • Try to force push and learn about protected branches in GitLab
  • Steps:
    • Run git push -f in the master branch
    • Explain the error
    • Navigate into GitLab > Project > Settings > Repository
    • Temporarily unprotected the master branch
    • Run git push -f again
    • Protect the master branch again and discuss with the trainer

Read more about protected branches and why they are useful here: https://about.gitlab.com/2014/11/26/keeping-your-code-protected/

Possible use cases for protecting branches:

  • Releases are tagged from the master branch, no-one should be allowed to modify this history at any point in time.
  • In case you are supporting older versions, you'll probably have release/x.y or support/x.y branches. No-one is allowed to override anything in there.

In case a commit gets lost, either being deleted or the history is rewritten, the corresponding release tag points to nothing. Meaning to say, that release is broken.

Therefore it is advised to protect sensitive branches in the project's repository.

Hint: Project maintainers can edit the settings. If you have the requirement to make this a permanent setting as administrator, you can e.g. use the REST API to always force this setting with a small script and cronjob.

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

More Hints: Force Push in shared branch, what now?

  • You pulled the 3 commits now being squashed in master.
  • Your colleage force pushed the branch with 1 squashed commit.
  • You cannot rebase anymore.
  • Instead, you need to reset your local branch to the remote branch base.

Discuss this with the trainer.

$ git fetch
$ git checkout feature/new-backend
$ git reset --hard origin/feature/new-backend

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

More Hints: Delete Remote Branches

You have learned that you can create remote (feature) branches. But what if you want to delete such branches?

git push origin <branch> is short for git push origin <localbranch>:<remotebranch>.

Pushing NULL into a remote branch will delete it.

git push origin :<remotebranch>

Hint: You can delete branches in GitLab/GitHub too.

~~~SECTION:MAJOR~~~ Git Workflows

NETWAYS Training

Lab 8.9: Delete remote branch

  • Objective:
    • Delete remote branch
  • Steps:
    • Change into $HOME/training
    • Create or identify a remote branch feature/docs-wrong-name
    • Delete the remote branch

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

9 Git Integrations

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

IDEs

  • Atom
  • JetBrains PHPStorm, ...
  • Visual Studio
  • Eclipse
  • vim

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Atom

  • Line diffs, tree view, etc.
  • Default editor for command line commits
  • GitHub integration with Pull Requests

Atom

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

JetBrains

  • Status and change tracking
  • Add, commit, fetch, push, pull, etc.
  • GitHub integration

PHPStorm

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Visual Studio

  • Git for Windows bundled in 2015+
  • Native GUI integration

Visual Studio

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Puppet Environments

  • Multiple Puppet environments (prod, stage, ...)
  • Git branches for each environment
  • Different module versions and configuration for each environment
  • Automated deployment with r10k

r10k is a toolset provided by Puppet for deploying environments and modules.


Reference blog post (German): https://blog.netways.de/2014/11/07/git-workflow-bei-puppet-mit-r10k/

Puppet environments and r10k are part of the "Scaling Puppet" training: https://www.netways.de/en/events_trainings/puppet_trainings/scaling_puppet/

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Puppet Environments: r10k

r10k

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Git Hooks

  • Client or server-side scripts
  • Conditional execution (update, post-receive, etc.)
  • Integration with external tools
  • Web hooks available in GitLab, GitHub, etc.

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Client Hooks

  • Useful to
    • Preserve file modes
    • Check code style/syntax

  • pre-commit
  • pre-rebase
  • post-checkout
  • post-merge

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Server-side Hooks

  • Deny non-fast-forward pushes
  • Check branch permissions
  • Notify external tools (Mail, IRC, Jenkins, etc.)

Note: Not available with GitLab CE.


Overview

  • pre-receive
  • post-receive
  • update

Server-side hooks receive the arguments through STDIN in the following format:

oldref newref refname

Example:

aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master

~~~SECTION:MAJOR~~~ Git Integrations

NETWAYS Training

Web Hooks

  • HTTP Callback
  • Limit on event, e.g. push, merge, comment, etc.
  • Integrate CI services
  • Integrate external ticket systems
  • Chat bot integration
    • Rocket.Chat
    • Slack

Reference example (German): https://blog.netways.de/2016/06/29/gitlab-webhooks/

Travis CI does not integrate well with GitLab, they prefer GitHub.

Jenkins

Jenkins builds with web books: https://github.com/jenkinsci/gitlab-plugin/wiki/Setup-Example

Rocket.Chat

Rocket.Chat web hook integration: https://rocket.chat/docs/administrator-guides/integrations/gitlab/

Slack

Official GitLab Slack integration: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/project/integrations/slack.md

Try it out

Both Rocket.Chat and GitLab are available at NWS:

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

10 Continuous Integration

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Continuous Integration

  • Immediately test changes in feature branches
  • Code quality checks
  • Deploy branches into environments (dev, staging, production)
  • Visualize failures and notify users
  • Reporting

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Jenkins

  • Standalone self-hosted server (Java)
  • Trigger build jobs from git commits
  • Start build jobs from specific branches (e.g. daily snapshots)
  • Run tests which update your ticket system
  • Build packages and deploy them into your repository

Git CI Jenkins

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Continuous Integration: Jenkins

  • Standalone self-hosted server (Java)
  • Trigger build jobs from git commits
  • Start build jobs from specific branches (e.g. daily snapshots)
  • Run tests which update your ticket system
  • Build packages and deploy them into your repository

Git CI Jenkins

Adding Jenkins to your Git workflow requires additional toolsets.

A Git hook configured on the git server is able to trigger a Jenkins job. It passes the git commit id to the Jenkins job which runs a local git checkout and further job actions afterwards, for example tests and/or package builds.

Vice versa a Jenkins build job hook needs to update the ticket system. This will notify the user of an unsuccessful run immediatly allowing further actions.

The issue id can be for example passed from the git hook into the Jenkins job. This requires strict rules on git commits - a git commit without any referenced issue id will destroy this workflow.

Define and document a git commit format specification and enforce this to your users. In addition to that install a git hook on the server which rejects a git push when the commit is missing an issue id.

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Travis CI

  • Cloud based CI system
  • Supports Ubuntu 14.04 LTS and macOS
  • Code and unit tests
  • Test multiple PHP, Perl, etc. versions at once
  • Easy to integrate into GitHub projects
  • De-facto standard for Open Source projects hosted on GitHub

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Travis CI Example

$ cat .travis.yml

language: php

php:
  - '7.1'

branches:
  only:
  - master

script:
  - php phpunit.phar -c modules/test/phpunit.xml --verbose

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI

  • Build Pipelines
  • Natively integrated into GitLab
  • Invokes GitLab Runners
  • Local container registry for runners

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI: Introduction

  • .gitlab-ci.yml configuration file in Git repository
  • Runner is triggered on specific events, e.g. git push
  • Jobs can be run on-demand
  • Built-in and external runners
  • Container registry enabled for the project (optional)

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab Runners

  • Written in Go
  • Linux/Unix, macOS, Windows, Docker support
  • Run multiple jobs in parallel
  • Run jobs locally, in Docker containers, remote via SSH
  • Can run Bash, Windows Batch/Powershell

Documentation reference: https://docs.gitlab.com/runner/

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab Runners: Installation and Configuration

  • Separate server
  • Installation via package repository
  • gitlab-runner register

Note: This is not needed for the built-in docker executor runner.

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI: Container Registry

  • Enable the Container Registry (administration server setting)
  • Enable the Container Registry for the project
  • Advanced usage only

```

vim /etc/gitlab/gitlab.rb

registry_external_url 'https://gitlab.example.com:5000'

gitlab-ctl reconfigure

```

Documentation References:

https://docs.gitlab.com/ce/user/project/container_registry.html https://docs.gitlab.com/ce/administration/container_registry.html

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI: Docker, Containers - how to use it

  • Run an application in an isolated environment
  • Layered images providing additional libraries and tools, e.g. base linux, mysql, apache, ruby
  • Start container, run tests, return results
  • Light-weight and fast, can run on each Git push
  • Reliable same run each time, container is destroyed on stop

Documentation References:

https://docs.docker.com

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI: Docker and CI Runners

GitLab CI Runners Docker

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Lab 10.1: Inspect CI Runner settings

  • Objective:
    • Inspect CI Runner Settings
  • Steps:
    • Navigate to Admin > Overview > Runners
    • Inspect the token
    • Check existing runners

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI: Configuration in .gitlab-ci.yml

  • image as container base image
  • services which should be running
  • all_tests as job name

Example:

image: alpine:latest

all_tests:
  script:
    - exit 1

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Lab 10.2: Create .gitlab-ci.yml

  • Objective:
    • Create CI configuration for the training project
  • Steps:
    • Create the .gitlab-ci.yml file in the training directory (vim, nano, etc.)
    • Add image: alpine:latest to specify base image
    • Add job all_tests with script as array element, which itself runs exit 1

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Lab 10.3: Push to GitLab

  • Objective:
    • Add .gitlab-ci.yml to Git and push to GitLab
  • Steps:
    • Use git add .gitlab-ci.yml and commit the change
    • Push the commit into the remote repository
    • Navigate to the project into CI / CD and verify the running job
  • Bonus
    • Modify the exit code to 0, add, commit, push and verify again

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Lab 10.4: Practical Example for CI Runners: Preparations

  • Objectives:
    • Prepare container to convert Markdown to HTML
  • Steps:
    • Modify .gitlab-ci.yml and add a before_script section after the image section
    • Update apk package manager and install python and py-pip packages
    • Use pip to install the markdown and Pygments libraries
    • Commit and push the changes

Example:

before_script:
  - apk update && apk add python py-pip
  - pip install markdown Pygments

The base image uses Alpine Linux which has a very small installation size and footprint.

In contrast to the "typical" Linux distribution containers, this allows for faster tests and deployment scripts.

If your build pipeline involves specific distributions, choose the best and reliable container distribution you prefer.

Alpine uses its own package manager. This exercise installs

  • Python and its package manager pip
  • Markdown conversion packages for Python

Reference: https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Lab 10.5: Practical Example for CI Runners: Create Docs

  • Objective:
    • Create HTML docs from Markdown
  • Steps:
    • Add a new markdown and use script to generate README.html
    • Add artifacts with paths pointing to README.html. Expires in 1 week
    • Commit and push the changes, then download and view the README.html file in your browser

Example:

markdown:
  script:
    - python -m markdown README.md > README.html
  artifacts:
    paths:
    - README.html
    expire_in: 1 week

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Lab 10.6: Practical Example for CI Runners: Update Docs

  • Objective:
    • Add what you have learned so far into README.md and generate docs
  • Steps:
    • Edit README.md
    • Commit and push changes
    • Download and view the README.html file in your browser

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI: .gitlab-ci.yml Templates

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab CI: Variables

  • Variables accessible in the Job container
    • Pre-defined environment variables
    • User-defined project variables
    • .gitlab-ci.yml defined variables

https://docs.gitlab.com/ce/ci/variables/


~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab Pipelines

Code => Build => Test => Deploy

  • Encapsulate QA steps into pipelines
  • Visible which pipeline is affected
  • Halt if one step is failing
  • Direct feedback to developers
  • CI and CD
    • Continuous Integration
    • Continuous Delivery

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

GitLab Pipelines: Example

GitLab Pipelines

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Lab 10.7: CI: Pipelines

  • Objective:
    • Build a job pipeline with stages

Example:

stages:
  - test
  - deploy

all_tests:
  stage: test

markdown:
  stage: deploy

~~~SECTION:MAJOR~~~ Continuous Integration

NETWAYS Training

Jobs, Stages, Pipelines

  • Stage pipeline

  • Failure: Edit the first job to exit 1.

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

11 GitLab Hints

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Issues, Merge Requests, Charts

  • Issues applied to task workflow
  • Merge requests with branches "ready to merge"
  • Overview charts and boards for better organization
  • Your "own" development and operations workflow

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Issue Boards

  • Plan, organize, visualize workflows for releases
  • Kanban or Scrum board
  • Filter by label
  • Drag&drop issues

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Lab 11.1: Use the Issue Board

  • Objective:
    • Use the Issue Board
  • Steps:
    • Navigate to Issues and create a new issue Learn about the issue board
    • Navigate to Issues > Board
    • Choose to create To Do/Doing labels
    • Drag the issue from Backlog to To Do to Doing to Closed
    • Explain the changed labels and workflow

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Web IDE

  • Write code in the browser
  • Commit changes
    • which triggers CI pipelines to provide immediate feedback
  • Syntax highlighting
  • Auto-completion
  • Under heavy development, use 11.2+

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Lab 11.2: Update README.md with the Web IDE

  • Objective:
    • Use the Web IDE to write documentation and verify CI results
  • Steps:
    • Navigate to Repository > Files and click Web IDE
    • Select README.md from the tree
    • Edit the file, use the live preview
    • Commit the changes, select Commit to master branch
    • Select the rocket icon on the right, check the pipeline status

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Project Snippets

  • Keep notes outside of the repository
  • Code snippets
  • URL can be shared

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Project Wiki

  • Additional documentation for the project
  • Supports Markdown (default), RDoc, Asciidoc
  • Live preview

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Project Graph

  • History lines in a graph
  • Similar to tig and GUI clients

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Administration

  • Default Settings
  • Stats, Versions, Users
  • Permission Management
  • Monitoring, System Hooks
  • Appearance and Customizing

Explore this with the trainer and discuss specific topics.


~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Administration: Appearance and Customization

Navigate to Admin > Appearance.

  • Custom header logo
  • Login page text
  • New project guidelines
  • Custom Themes (SASS, CSS)

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Best Practices

  • HTTPS only
  • Performance tuning
  • Monitoring
  • Troubleshooting

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Secure HTTPS environment

  • Use Let's Encrypt or own certificates
  • Enable HTTPS in GitLab/Nginx
  • Forward all HTTP requests to HTTPS

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Performance Tuning

  • 4+ GB RAM make GitLab happy
  • Move PostgreSQL 9.2+ to a dedicated host

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

Monitoring

  • HTTP(S)
  • PostgreSQL
  • Redis
  • Background daemons
  • Logs

In addition to that, you can query the REST API.

Example for CI runners: /api/v4/runners/all


An example for querying the REST API is described in this blogpost: https://blog.netways.de/2018/09/05/icinga2-gitlab-health-check/

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

GitLab API

  • Automation
    • Users, Groups, Projects, etc.
  • Handle Merge Requests
  • Jobs, Pipelines, CI
  • ...

You can manage nearly everything.

~~~SECTION:MAJOR~~~ GitLab Hints

NETWAYS Training

GitLab Applications

GitLab provides an extensive list of external applications:

https://about.gitlab.com/applications/

  • CLI clients
  • API clients
  • GUIs
  • ...

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

12 Git Hints

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Best Practices

  • Ensure to set user configuration (name, email)
  • Always keep good commits in mind
  • Do not commit all changes at once
  • Work with branches if applicable
  • Avoid merge commits, fetch and rebase your history

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Hints for details and tests

  • verbose mode for cli commands: -v
  • test-drive but don't change anything: --dry-run

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Debugging

  • git bisect
    • Binary search for commits causing a bug
  • git blame
    • Open editor with commit/author/date addition
  • git grep
    • Search for pattern in commit history

git bisect can be used to invoke a binary search to find the commit that introduced a bug.

git blame prints a file and each line changed by the latest commit and author.

git grep prints all lines from the commit history matching a pattern.

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Lab 12.1: Use Git Blame

  • Objective:
    • Use git blame
  • Steps:
    • Pick a file from your local git repository
    • Use git blame filename
    • Explain the line prefix and its meaning

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Restore deleted file

This is where git rev-list comes in handy.

Find the last commit which affected the given path.

git rev-list -n 1 HEAD -- <filepath>.

Then checkout the version at the commit before using the caret ^ symbol.

git checkout <deletingcommit>^ -- <filepath>

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Apply and Create Patches

Manually apply and create Git patches.

  • git apply: apply git patch
  • git am: read patches from STDIN and apply them
  • git format-patch: create git patch

git apply applies a git patch file to your working tree.

git am reads git patches from your mailbox by default. It also can be used to read patches from the shell's STDIN (e.g. curl downloading a patch from the web). The -s option allows you to sign off the patch, for example if you have reviewed the patch from an external committer.

git format-patch requires either a negative number of commits from current HEAD or a git commit id. It will then create numbered patch files for all required commits.

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Git Submodules

  • Pointer to a specific commit in remote repository
  • Independent sub directory with own .git repository

Example:

$ git submodule add https://github.com/Icinga/puppet-icinga2
$ git status
  new file: .gitmodules
  new file: puppet-icinga2

$ git submodule
  ad5e309... puppet-icinga2 (heads/master)

Fresh clone:

$ git clone --recursive ...

$ git clone ...
$ git submodule init && git submodule update

Note: Git submodule removal is not trivial.

https://davidwalsh.name/git-remove-submodule

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Git Subtree

  • Merge an external git repository history
  • Directory in your main repository (--prefix)
  • Squash the remote history into one commit
  • No external dependency (e.g. git submodule repository not available anymore)
  • No local configuration - document the external reference

Example for imported Puppet modules:

$ git subtree add --prefix modules/icinga2 \
https://github.com/Icinga/puppet-icinga2 HEAD --squash

$ git subtree pull  --prefix modules/icinga2 \
https://github.com/Icinga/puppet-icinga2 HEAD --squash

Adding a git subtree requires

  • Existing directory tree except for the one created
  • Git clone URL
  • The branch which should be added (use master as default)
  • Optionally --squash all commits into one

Example:

git subtree add --prefix clippy.js https://github.com/smore-inc/clippy.js.git master --squash

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Git Aliases

  • Shortcuts
  • Custom commands
  • "Simulate" SVN commands

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Lab 12.2: Add an alias for git diff

  • Objective:
    • Add an alias for git diff
  • Steps:
    • Edit the $HOME/.gitconfig file
    • Add a new [alias] section if not existing
    • Add d as an alias for diff

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Git Attributes

  • Control the file line ending mode on different OS
  • Ignore specific files on git archive

Example:

$ vim .gitattributes

modules/** eol=lf

~~~SECTION:MAJOR~~~ Git Hints

NETWAYS Training

Migration from SVN

  • Preserve history?
  • No history, copy source and start fresh