Linux

Working with different Git configs

2017-03-10 DSP2017, Git, Linux, macOS No comments

Short introduction

Sometimes people need to specify multiple values for single .gitconfig file or they want to share just part of the configuration between two machines. There are different approaches for that. I can show you mine.

Different configs for different Operating Systems

On my private computer, I use Linux. I use Git for my private projects and I use my private e-mail address there. At the same time, I use Git at work on macOS with exactly the same Git configuration, but with a different e-mail address. How to deal with that?

In my .gitconfig file, I set my private e-mail address, which is used by default. In my .zshrc file, I created two aliases:

alias setupGitPersonal="git config --global user.email \"piotr@wittchen.biz.pl\""
alias setupGitForWork="git config --global user.email \"piotr.wittchen@sap.com\""

Hint: If you want to configure more stuff than just an e-mail, you can do it in the appropriate alias or you can create separate shell scripts for that and place them in /usr/local/bin/ directory.

Then, on Linux, I don’t have to do anything and my private e-mail address is used out-of-the-box.
On macOS, I do the following trick in .zshrc file:

if [ `uname` = "Darwin" ]; then
  setupGitForWork

  # rest of the macOS config goes here...
fi

After that, every time I start terminal on macOS, it automatically sets up my e-mail address to the one I use at work and keeps my .gitconfig file updated.

Hint: If you don’t use zsh, instead of .zshrc file, edit .bashrc file.

Different configs for the same OS on two machines

If you’re using different configs on the different machines with the same OS, you can try another trick. Create configuration file – e.g. .machine_name in your home directory. Setup one name on one machine and another name on a different machine. Next, include this file in your .zshrc or .bashrc file, perform appropriate check and load different settings basing on variable name.

. ~/.machine_name

if [ $machineName = "workMachine" ]; then
  setupGitForWork
else
  setupGitPersonal
fi

Contents of the .machine_name file are simple:

machineName="workMachine"

Different configs on the single machine with one OS

In such case, we are supposed to perform the manual switch. We can use aliases provided above. When we want to have personal settings, we can open terminal and type setupGitPersonal. When we want to apply work settings, then we can type setupGitForWork.

Summary

As we can see, keeping different configs for different machines or operating systems and changing them depending on our needs is not so hard. I hope these ideas will help you to manage your configs.

Control Spotify on Linux like a hacker

2017-03-05 DSP2017, Linux, Open source, Python No comments

Recently, I created a tiny script called spotify-cli, which allows you to control Spotify on Linux from terminal. It’s inspired by shpotify, which is a shell script doing similar things, but on macOS. My script is written in Python and uses dbus under the hood, which allows to communicate with bus daemon to pass messages between applications. I used pactl for controlling the system sound.

You can install spotify-cli as follows via wget:

sh -c "$(wget https://raw.githubusercontent.com/pwittchen/spotify-cli-linux/master/install.sh -O -)"

or via curl:

sh -c "$(curl -fsSL https://raw.githubusercontent.com/pwittchen/spotify-cli-linux/master/install.sh)"

After that, you can just type spotify-cli in your terminal.
You can use spotify-cli with the following parameters:

--help, -h          shows help
--status            shows status (currently played song name and artist)
--play              plays the song
--pause             pauses the song
--playpause         plays or pauses the song (toggles a state)
--next              plays the next song
--previous, --prev  plays the previous song
--volumeup          increases sound volume
--volumedown        decreases sound volume

That’s it! Happy listening!

Source code of the project can be found at https://github.com/pwittchen/spotify-cli-linux.

Lifting quality of a shell script

2016-11-30 Bash, Linux No comments

Introduction

In release cycle of our team at work, we need to perform so-called system tests. In order to do that, we need to log into Artifactory, search for the latest release package, check if it’s up to date, download it, unzip it, install internal configuration recipe, compile, initialize & run it. Not all of that can be easily automated, but I thought that at least searching & downloading phase can be done from the terminal in a semi-automated way. That’s why I created ydownloader shell script.

Writing a script with unit tests and continuous integration

I’m not an expert in shell scripting, so I also wanted to learn more about it. In addition, I wanted to apply best software development practices in that script. Someone can say that in the case of a simple shell script proper engineering may be a superfluity, but in my opinion, the simplicity of the project is not an excuse for doing it the right way. Especially, if we want to use it in the future. That’s why I divided this script into smaller functions, added command line arguments handling and help for the users. Moreover, I added unit tests with shunit2 (yes, we can write unit tests for the shell scripts) and continuous integration with Travis CI server. In the “Clean Code” book, we can read that code without unit tests is not clean by definition. After dividing script into smaller functions, it was much easier to test it. My script is accepting command line arguments, so I needed to do the following trick to make it testable and include it in the testing script:

if [ "$TEST_MODE" == "" ]
then
  TEST_MODE=false
fi

if [ "$TEST_MODE" = false ] ; then
  # parse command line arguments here...
else
  echo "TEST_MODE enabled"
fi

Then, I could write unit tests:

TEST_MODE=true
. ./ydownloader # load script to be tested
echo "RUNNING UNIT TESTS..."

testCutLastChars() {
  # given
  valueToCut='testString'
  expectedValue='testStri'

  # when
  actualValue=$(echo $valueToCut | cutLastChars 3)

  # then
  assertEquals $expectedValue $actualValue
}

# more tests goes here...
. ./shunit2/shunit2 # load shunit2

There are also other solutions for unit testing like bats, assert.sh and others. We can choose what we like. We can also use additional tools like shunit2-colorize to make our console output of shunit2 tests look like a rainbow if we are not fans of monochromatic terminal. Moreover, we can use static code analysis tools for shell scripts like shellcheck.

In addition, I prepared simple install script, which allows to install script locally via curl or wget. Of course, project has sufficient documentation in README.md file.

Short recap

It was really nice coding exercise. Now, I feel much more comfortable with shell scripting, but there’s still a lot to learn. I recommend trying applying a similar approach in your scripts if you haven’t done it yet.

If you want to browse complete project, check it out in my repository: https://github.com/pwittchen/ydownloader.

Converting audio CD to *.mp3 files on Linux

2016-01-24 Linux, Ubuntu 2 comments

Not all songs are available on Spotify and sometimes we need to rip music from audio CDs in order to listen it on our computer or mobile device.

There are several approaches to do that. Here is mine:

Get RipperX:

sudo apt-get install ripperx

Put audio CD into your computer. Open RipperX, select All Tracks and check Rip to WAV option. Set quality of output files via “Config” option and names of the tracks if your want. Press “Go!”. After conversion, you should have directory with ripped *.wav files in your home directory.

Get SoundConverter:

sudo apt-get install soundconverter

Run it and convert *.wav files to *.mp3. You can choose another output format if you want.

Get EasyTag:

sudo add-apt-repository ppa:amigadave/ppa
sudo apt-get update
sudo apt-get install easytag

Open EasyTag and set appropriate tags for your *.mp3 files. You can also set CD cover as image for each file.

You are done!

Now, you can import files to your favorite music player. I’m using Spotify right now, where I can select local files, create a playlist and even sync it with my mobile device if my computer is in the same WiFi network. It’s very handy option.

I hope, this short tutorial will be helpful for you.

3 questions about your Git repository

2015-12-28 Git, Linux, Open source, Python 3 comments

Introduction

Can you answer the following questions about your Git repository?

  1. Does development branch has all changes from master branch?
  2. Is your gitlog a crap?
  3. How old are your branches?

If not, but you want to know answers, you’re lucky, because I prepared 3 simple scripts for you, which can help to find it out.

Does development branch has all changes from master branch?

git-branch-comparator is a python script, which checks if development branch has all changes from master branch in Git repository.

Another, easier way to accomplish the same task suggested in comments by Mike (thanks!) is to call simply:

$ git pull
$ git branch --contains master --no-merged development

When we are working in a Git Flow and critical bug occurs on production, sometimes there is a necessity to create so called hot-fix. We can create separate branch from master branch for this hot-fix and then merge it into master branch or we can commit a change on master branch. Second option is not recommended. After that, we have to remember to merge master branch into a development branch to have our hot-fix in a development version as well and avoid merge conflicts in the future.

This python script checks, if all changes made on master branch were also merged into development branch to keep those two branches consistent. We can add it as a job into Jenkins CI server and monitor branches consistency. In addition, release jobs can depend on that job and we can avoid merge conflicts or project unstability before release.

source code & documentation: https://github.com/pwittchen/git-branch-comparator

Is your gitlog a crap?

craplog is a python script, which checks if git git log of the given project is crappy or not. Right now, script is very simple. It just checks if more than half of the commit messages are good. Commit message is considered as good, when it contains more than two words. Of course, this is not the only condition determining the quality of the commit message, but this is early beta version of the script and can be improved later.

I’ve read a discussion in one of the pull requests to Linux kernel. It made me think about quality of Git commit messages. Of course, Linux kernel is a specific project and has its own standards. Maybe not all of these standards will be valid for a simpler or less complicated projects. Nevertheless, a lot of people don’t pay attention to git commit messages. They put crappy stuff inside them like random letters and numbers or stupid expressions, which has no specific meaning, aren’t related to the project or aren’t informative enough. In my opinion, good git log is one of the factors determining good quality of the project. Sometimes, we need to browse log to find some changes or analyze project history in order to fix a bug or find important information. It’s easier to do it, when git log is good. I’ve made some of the mentioned mistakes in the past, but I try to avoid them now.

source code & documentation: https://github.com/pwittchen/craplog

How old are your branches?

git-aged-branches is a shell script showing git branches of defined repository with age of their last commit. It works on Mac OS X, Linux and can be helpful while investigating old Git branches to delete. This script does not delete anything! It’s just for informational purposes.

source code & documentation: https://github.com/pwittchen/git-aged-branches

Summary

I hope, some of you will find these tools useful and maybe they’ll solve your current problems or help to improve quality of your projects.
If you would like to know more details about mentioned projects, check instructions how to use them and their source code, visit linked repositories on GitHub.

Note: Any feedback, new issues or pull requests are appreciated!

Using tmux

2015-09-20 Linux No comments

What is a terminal multiplexer? It lets you switch easily between several programs in one terminal, detach them (they keep running in the background) and reattach them to a different terminal. And do a lot more.

– tmux.github.io

ss-tmux1

Tmux feature, which I find very useful is tiling terminal window. We can have several tiles with different terminals within a single terminal window.

How to use tiling?

First, we need to install tmux:

$ sudo apt-get install tmux

Then, we need to start it:

$ tmux

When we are inside tmux, we can execute its commands. It’s good to check full list of tmux key bindings.

Default initial key binding for different commands is Ctrl+B. When we hold Ctrl and then press B, we can press next key for specific command. It’s tricky and it isn’t intuitive at the first time.

For example, if we want to split terminal window vertically, we need to do the following thing:

Start tmux, Hold Ctrl, press B (while holding Ctrl), release buttons and press % key (equivalent to Shift+5).

If we want to split terminal window horizontally, we need to do the following thing:

Start tmux, Hold Ctrl, press B (while holding Ctrl), release buttons and press " key (equivalent to Shift+' – code for ' sign is 47 for xdotool).

We can create any tile configuration we want like in i3 windows manager.

tmux-tiles

If we want to switch between tiles, we need to use the following key combination:

Hold Ctrl, press B (while holding Ctrl), release buttons and press O key (“O” letter – not zero).

Creating 4 tiles automatically

Popular terminal windows configuration is 4 tiles (2 columns and 2 rows). We can split windows horizontally or vertically pretty fast with default shortcuts, but creating layout consisting of 4 tiles requires some clicking. I’ve written a simple script, which generates such layout for us automatically and saves the time.

First, we need to install xdotool:

$ sudo apt-get install xdotool

Next, we can create file named tmux-4tiles, set its chmod to 777 and save it in /usr/local/bin/ directory.

File should have the following content:

#!/bin/bash
# generates 4 tiles in tmux (requires tmux and xdotool)
xdotool key ctrl+b shift+5 && xdotool key ctrl+b shift+48 && xdotool key ctrl+b o && xdotool key ctrl+b shift+48 ctrl+b o ctrl+b o ctrl+b o && clear

It’s also available in my dotfiles on Github.

When, we are done, we can enter tmux:

$ tmux

and run the script:

$ tmux-4tiles

After that, we’ll get the following layout:

tmux-4tiles

We can automate generating different layouts for our purposes in the same way.

Recap

In my opinion, tmux is very useful tool for people working with terminal who want to have organized windows in an elegant way.

References

Gnome Classic desktop environment on Ubuntu

2015-08-23 Linux, Ubuntu, UI No comments

Introduction

I was tired of non-minimal and quite slow Unity desktop environment. Of course, I performed a few tricks to make Unity faster, but still I wasn’t satisfied enough. I checked out different desktop environments. I wanted to have clean, minimal and productive environment. I like top panel from Unity as well as HUD and many workspaces. The last thing is quite common among different desktop environments.

New desktop environment

I decided to choose Gnome classic. It’s fast, clean, minimal, works easily with Ubuntu, has top panel and is configurable.
My current desktop looks like this:

screenfetch-2015-08-18-11_00_51

Unfortunately, I don’t have HUD like in Unity, but after a few days I got used to that. I also turned off all animations and visual effects. Everything works smoothly and looks much better than Unity. In the current configuration I have: Z Shell, Oh-my-zsh, dmenu, Numix Theme and Numix Circle Icons. In addition, I have the same indicators, which I had earlier on Unity and they work fine. I just needed to adjust look of Spotify icon in top panel. Moreover, Gnome Pie was installed later as additional launcher and media controller.

Installation of Gnome Classic and Numix Theme

I’ve installed Gnome Classic as follows:

sudo apt-get install gnome-session-fallback

Then, I installed Gnome Tweak Tool and Unity Tweak Tool:

sudo apt-get install gnome-tweak-tool unity-tweak-tool

and Compiz Config Manager with plugins:

sudo apt-get install compizconfig-settings-manager compiz-plugins-extra

Please note: to enable alt+tab in gnome classic fallback, open the manager and navigate to window management and check application switcher (previously disabled).

After that I could install Numix icons and theme:

sudo add-apt-repository ppa:numix/ppa
sudo apt-get update
sudo apt-get install numix-gtk-theme numix-icon-theme numix-icon-theme-circle

I’ve set appropriate theme and icons via Gnome Tweak Tool. After reboot of the computer changes should be successfully applied.

Detailed information about configuration

If you’re interested in detailed configuration information, you can check out my dotfiles at: https://github.com/pwittchen/dotfiles and system configuration at: https://github.com/pwittchen/ubuntu-config.

To generate system information for the screenshot I used screenfetch script. I didn’t worked correctly for gnome-session-fallback, but I’ve made small contribution on GitHub and now it’s fine.

Wallpaper

You can find wallpaper from the screenshot at papers.co website.

Alternatives & Resources

I’ve spent some time on analyzing alternative desktop environments and Linux based operating systems, which looks good.
Below, you can find my collection of information and resources. Maybe some of them will be better for your specific needs.

Interesting Reddit channels:

Selected Linux based systems with interesting user interface:

Selected Linux window managers:

Recap

After switching from Unity to Gnome my desktop is much more better, faster and cleaner. After mastering popular shortcuts for managing programs, windows, workspaces, etc. we can work very efficiently. Gnome Classic is fine for me right now, but maybe other WM will be better for you. It depends on your personal preferences. I’m gonna use Gnome Classic for some time and maybe I’ll try other environments in the future. For sure I won’t go back to Unity if it don’t evolve.

Geary – neat e-mail client for Linux

2015-08-01 Linux, Ubuntu No comments

Overview

geary-home1
I was using web interfaces for e-mail for a long time, but I wanted to give a try a desktop clients for Linux. I was searching for a quite simple solution with almost zero configuration, which I can use for my private and work e-mail accounts. Geary seems to be quite good choice. It is clean, easy to use and pretty neat e-mail client for Linux. It integrates with Unity on Ubuntu and display system notifications informing about new message. Unfortunately, we need it running in order to see notifications, which is small drawback. Nevertheless it works quite good, so I’m going to give it a try.

geary-10-750x404

Install it with the following command:

sudo apt-get install geary

Pros & cons

Here is my list of pros and cons of this software.

Pros

  • almost zero configuration
  • clean & neat interface
  • multiple e-mail accounts
  • integration with the system and notifications informing about new messages
  • recipient suggestions without importing contacts
  • limiting range of downloaded messages – e.g. we can download everything or just messages from last 2 weeks
  • it’s open-source: https://github.com/GNOME/geary – as we can see on GitHub, it’s actively developed
  • it’s free

Cons

  • almost zero configuration, which may be drawback for some people 😉
  • recipient suggestions does not work with all contacts (I suppose it may be connected with range of downloaded messages)
  • notifications works only when application is running
  • no contact list
  • no calendar available

Interesting fact

It’s written in Vala language.

Recap

Not all mentioned cons are really bad. As it’s just an e-mail client, it doesn’t need to have contact list or calendar. It’s additional functionality, but there may be some problems while working with systems like MS Exchange or something similar where user need to confirm presence on appointment at work or something like that. To sum up, regardless of a few drawbacks, I can tell that Geary is really nice piece of software, which can be used on daily basis by people who like simple solutions.

Routing internet via SSH tunnel on Raspberry Pi

2015-07-25 Linux, Network, Raspberry Pi, Ubuntu No comments

Problem overview

Recently I had the following problem. I wanted to connect to my Raspberry Pi from my laptop. For some reason WiFi dongle on the Raspberry Pi was not working properly. Moreover, I had only one slot available for the LAN connection via cable to the router, so I could connect only one device this way. I wanted to have reliable and fast internet connection on my laptop, but via WiFi it’s quite slow.

To sum the things up two goals were clear:

  1. to have fast and reliable internet connection on the laptop
  2. to be able to connect to Raspberry Pi via SSH in order to control it from the laptop

Enabling SSH server on Raspberry Pi

Before we proceed to the next steps, we need to enable SSH server on Raspberry Pi if we haven’t done it in the past.
In order to do that, we need to go through the following steps:

  1. login with the following information when prompted:
    username: pi
    password: raspberry
  2. type the following command in the terminal: sudo raspi-config
  3. then navigate to ssh
  4. hit Enter
  5. select enable ssh server
  6. reboot device
  7. we are done!

Sshuttle to the rescue

I tried a few different approaches to solve described problem, but up to now the only one solution satisfies me well.
I’ve found discussion starting with the question How do I route my internet through a SSH tunnel? In this discussion, I’ve read about great tool called sshuttle, which works as a poor man’s VPN.

sshuttle is a transparent proxy server that forwards over a SSH connection and sets up a proxy by running Python scripts on the remote server. sshuttle can be run under the following conditions:

  • client machine or router is Linux-based, FreeBSD or Mac OS
  • administrative privileges on client
  • access to remote network via SSH
  • no administrator privileges on remote network
  • availability of Python on remote server

sshutle can be installed with the following command:

sudo apt-get install sshuttle

The basic command for running sshuttle with routing all traffic is:

sshuttle -r username@sshserver:port 0/0

Upon the execution of the command, a sudo password prompt will appear and subsequently the password to SSH account. No other details will appear except for a short message and return to shell upon failure. For more status messages, run sshuttle in verbose mode with the -v flag.

In this example all internet traffic except DNS is routed through the VPN. -r flag denotes the remote hostname and optional username and port that follows in the above example. 0/0 is short for 0.0.0.0/0 that represents the subnets to route over the VPN. The usage of 0/0 routes all the traffic except DNS requests to the remote server. DNS tunelling is possible with the usage of -H flag.

Please read the man page (man sshuttle) for the details of options and modes under which sshuttle can run. For information about the concept and more examples, refer to the project page.

РAndr̩ Param̩s on askubuntu.com

Connecting to Raspberry Pi and tunneling internet connection

Luckily, all requirements were satisfied by Raspberry Pi and I could use it with sshuttle, so I’ve installed this software on my laptop.
I’ve connected to my local network through WiFi. Then, I’ve scanned network with nmap to find IP of Raspberry Pi.

nmap -sP 192.168.1.0/24

My local network consists of only few devices and each of them has a description, so I could easily identify IP of the Raspberry Pi.
Output looked like this:

Nmap scan report for livebox.home (192.168.1.1)
Host is up (0.0070s latency).
Nmap scan report for laptop-piotra-*samsung-ultrabook*.home (192.168.1.14)
Host is up (0.00011s latency).
Nmap scan report for pc36.home (192.168.1.17)
Host is up (0.018s latency).
Nmap done: 256 IP addresses (3 hosts up) scanned in 2.64 seconds

Right now, active devices in my network are: router, my laptop and Raspberry Pi, with IP: 192.168.1.17.
After that, I connected to Raspberry Pi with the following command:

sshuttle -r pi@192.168.1.17 0/0

It asked me about the local password and then about the password of remote host, which is raspberry by default and I saw the following message:

Connected.

Everything was fine. I was connected to Raspberry Pi and I was able to access internet through it.
Next, I opened another terminal window and tried to connect to Raspberry Pi via SSH.
Please note, that you have to enable remote access via SSH on Raspberry Pi before you try to establish connection.

When SSH is enabled, you can type the following command:

ssh pi@192.168.1.17

Then I typed correct password and I was connected!

Linux raspberrypi 3.18.7+ #755 PREEMPT Thu Feb 12 17:14:31 GMT 2015 armv6l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat Jul 25 01:49:00 2015 from pc36.home

Short summary

What is surprising tunneled internet connection was fast and stable when I was accessing it through Raspberry Pi.
When I was accessing internet directly through WiFi on the router it was really slow for some unknown reason.
Luckily, I’ve achieved my goals: I’ve fast internet connection on the laptop and I coul establish SSH connection with Raspberry Pi over local network.

Glances – light, but powerful system monitoring

2015-07-11 Devops, Linux, Python No comments

Recently, during reading information in The Art of Command Line repository, I’ve found Glances, which is a cross-platform curses-based system monitoring tool written in Python. It’s light, but very powerful tool, which allows to monitor your system. Moreover, it’s open-source.

glances

Before you install Glances, you need Python, PyPI and Psutil. If you want to run Glaces as a web server, you also need to install Bottle.
To monitor temperature of your CPU, install Pysensors.

You can install the most important packages with the following command:

$ sudo pip install psutil bottle pysensors

and then install Glances

$ sudo pip install glances

After that, just run glances

$ glances

If you want to run Glances as a web server, type:

$ glances -w

open the following address in your favorite web browser: http://<your_ip>:61208/ and replace <your_ip> with appropriate value.
If you just want to check it locally, you can use this address: http://127.0.0.1:61208/.