Portable Dev Server

Introduction

In my day to day use of computers, I prefer to operate and edit in Windows due to some of the software packages I use. Unfortunately, usually when I'm developing software, I prefer to compile and test in a Linux environment.

The Portable Dev Server fills this gap by providing an authentic linux shell to me in Windows.

Contents

Features

The network configuration is setup to be as flexible as possible. Whether I have Ethernet, WiFi or need to fall back to Mobile Phone / Cell Phone tethering, the Portable Dev Server is able to provide access for my PC. If I haven't used a WiFi network before, I tether the RPi to my phone and setup the WiFi access on that then reboot.

The power supply is also robust. It is uninterruptible, meaning that SD card corruption due to power-loss-during-write is elliminated.
The Portable Dev Server can also run with-out a 5V input source for a significant amount of time just like my laptop, meaning that my development environment can come with me.

Collaboration is also available. Any other user on the same network can be allowed to access the Portable Dev Server network-shared workspace and have SSH terminal access.

Hardware Setup

My current configuration is as follows:

• Raspberry Pi 3 Model B+
• 16GB Toshiba Exceria UHS-I / Class 10 MicroSD card (being upgraded now)
• LiFePO4wered/Pi+™ 5V UPS and Power Manager
• Cheap I²C OLED screen
• Ethernet Cable (when not using WiFi)
• 5V Source and MicroUSB Cable (for longer sessions)

I find the RPi3 B+ sufficient for the work I do on it as I use it purely headless via SSH, SAMBA and HTTP.
Depending on the type of work you intend to do, concider which version you'd prefer.
The RPi 4 certainly has a lot more power, but by all reports it is hard to keep cool and uses a significantly larger amount of power.

Up until now, I haven't had any issues with speed from the current MicroSD card, it is a little lacking in capacity.
I've recently bought a 128GB SanDisk Extreme card which is rated A2 class, which is apparently a rating of app performance rather than pure transfer speed. However, I want to install the new version of raspbian and build up the server from scratch, not just clone the existing card.

I'm writing this guide as I do this.

Operating System Installation

Raspberry Pi OS (formerly named Raspbian) used to require setup by download the operating system image, then flashing it to the SD card using belenaEtcher.

While you can still do it this way, Raspberry Pi Imager simplifies the process.

Using Raspberry Pi Imager, under 'Operating System' I chose the 'Raspberry Pi OS (other)' option, then 'Raspberry Pi OS Lite (32-bit)' selection.
I selected the SD Card which I had connected via a USB-SD card adapter and click Write.

Because I run the RPi 'headless', I next unplugged and replugged the SD card so that I could browse the card and add a blank file named "ssh" (no extension) onto the root directory of the card, which enables SSH remote access at first boot.

Connecting to the RPi

At this point, inserting the card applying power will boot the RPi.

Ensure that you connect an ethernet cable first and use the built-in usb power jack (the LiFePo4wered jack will disconnect the power to the RPi when it doesn't detect the correct software running - to be installed soon).

Once booted, the green status LED will be lit less often. You now need to determine the IP address of the RPi.

If you have only this Raspberry Pi connected, you can open the windows command line and type the ping raspberrypi /4 command, which will display something like this:

Pinging raspberrypi.local [192.168.1.4] with 32 bytes of data:
Reply from 192.168.1.4: bytes=32 time<1ms TTL=64
Reply from 192.168.1.4: bytes=32 time=1ms TTL=64
Reply from 192.168.1.4: bytes=32 time<1ms TTL=64
Reply from 192.168.1.4: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.1.4:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 1ms, Average = 0ms

From the above, we can see that the RPi is using IP address 192.168.1.4.
We now know what we need to connect to it for the first time!

To connect to the RPi from Windows, I use PuTTY which is freely available here.
The MSI Windows Installer option is the easiest way to install it.

Once installed, start PuTTY and you will be greeted with a configuration window. All that is needed is to type the IP address into the 'Host Name' box and click open.
The first time you conenct, you will get a popup which asks if the server is trusted - allow it and it won't bother you again.

You will now be asked who to login as. The default username is pi and the password (you wont see anything typing) is raspberry.

You are now logged into your Raspberry Pi via SSH (Secure SHell)!

System Configuration

Before progressing, it is a good idea when running a fresh install of linux to update everything. This is done using the following two commands:

sudo apt-get update
sudo apt-get dist-upgrade

Hostname & Raspi-Config

Let's rename the RPi so that it can be found among other RPis. This is done by modifying a few files as follows:

1. Replace raspberrypi in the hostname file... I'm using PortServer as the name for my pi:
   • sudo nano /etc/hostname
   • Pressing Ctrl+X then y then Enter saves and closes the file once the change is done.
2. Similarly, replace raspberrypi in the hosts file using this command sudo nano /etc/hosts.
3. Reboot the RPi using the command sudo reboot now.
4. You will need to reconnect using PuTTY (restart the session) after 30 or so seconds as the SSH session is closed upon shutdown.

To make the new name more visible to Windows, install SAMBA (file sharing) and the WIN binding to list the hostname in "Network" under Windows:

sudo apt-get -y install samba samba-common-bin
sudo apt-get -y install winbind

During the first install, I chose NOT to use the WINS settings from DHCP.
Your network is likely different, so decide on this yourself.

Add wins to the end of the line that begins with hosts: in the nsswitch.conf file: /etc/nsswitch.conf.

Reboot the RPi and reconnect.

Next we will setup some system settings in the configuration tool. Start it using the sudo raspi-config command.

In this menu, I'd recommend changing/setting the following:

1. WLAN Country and Timezone in the Localisation Options
2. User Password

SAMBA Shared Directory

Setup a shared directory so data is accessible on Windows as well. The following command makes a directory which is accessible by typing cd /share within Linux/SSH.

sudo mkdir /share/
sudo chmod 777 -R /share/

If you didn't change the hostname in the previous section, you still need to install SAMBA using the following command:

sudo apt-get -y install samba samba-common-bin

Next, we setup the new folder as a SAMBA share by adding the following info into the smb config file: /etc/samba/smb.conf.

[share]
Comment = Pi shared folder
Path = /share
Browseable = yes
Writeable = Yes
only guest = no
create mask = 0777
directory mask = 0777
Public = yes
Guest ok = yes

I also added the following into the network section of /etc/samba/smb.conf because I was getting an error when browsing the host shares from Ubuntu Failed to receive share list from server: invalid argument:

client min protocol = NT1
server min protocol = NT1
name resolve order = bcast host lmhosts wins

Set the SAMBA password for 'pi' and restart SAMBA with the new configuration using the following commands:

sudo smbpasswd -a pi
sudo samba restart

You should now see the 'share' folder under the RPi (\\PortServer\share\ for me).

LiFePO4wered/Pi+™ Daemon

The RPi needs to run a service which communicates with the LiFePO4wered/Pi+ hardware so that it can shutdown when the battery is running out.

The hardware (and software) also allows for additional information to be viewed within Python and the command line.

To install the service use the following commands:

git clone https://github.com/xorbit/LiFePO4wered-Pi.git
cd LiFePO4wered-Pi/
python2 build.py
sudo ./INSTALL.sh

Shutdown the RPi using the command sudo shutdown now, move the 5V input to the LiFePO4wered/Pi+ input, and press it's on button and reconnect via SSH.

Refer to the LiFePO4wered/Pi+ manual for more configuration options.

Flexible Networking

I set the RPi to prioritise an Ethernet connection. When one is connected at boot, the RPi shuts down the WiFi adapter.
If Ethernet is not available it looks for the known WiFi networks and attempts to connect to them if they are found.

When I don't have eitehr option available, I enable my iPhone hotspot, which the RPi knows, and my laptop can also be connected to this, giving me a portable solution

To achieve the Ethernet priority, the following code is added to the /etc/rc.local file before the exit 0 line.

# Disable WiFi if wired.
logger "Checking Network interfaces..."
if ethtool eth0 | egrep "Link.*yes" && ifconfig eth0 | grep "inet "; then
logger 'Disabling WiFi...'
ifconfig wlan0 down
else
logger 'WiFi is still enabled: Ethernet is down or ethtool is not installed.'
fi

If using a RPi4, the following will work instead.
Add a line such as /share/tooles/check_eth & before return 0 in rc.local, then the script you just pointed to should be created and filled with the following:

#!/bin/bash
# This script checks if eth0 can connect, if so disable WiFi
# Let system get up first, default wait: 30s:
# Check for initial delay on command line

if [ -z "$1" ]; then
  DELAY="30"
else
  DELAY="$1"
fi
sleep $DELAY

EthAddr=$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)
WiFiAddr=$(ip addr show wlan0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)

if [ -z $EthAddr ]; then
  echo "IP addr of eth0 not found, so use WiFi"
  echo "wlan0 IP=$WiFiAddr"
else
  echo "Using Ethernet! Switch off WiFi."
  echo "eth0  IP=$EthAddr"
  echo "wlan0 IP=$WiFiAddr"
  cmd="ifconfig wlan0 down"
  $cmd
fi

To add known WiFi networks for the WiFi option the wpa_supplicant file: /etc/wpa_supplicant/wpa_supplicant.conf.
Mine looks SOMETHING LIKE this.

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=AU

network={
ssid="iPhone"
psk="iPhonePass1234"
}

network={
ssid="HomeWifi"
psk="homePass9876"
}

network={
ssid="WorkWifi"
psk="workPass2143"
}

network={
ssid="FriendWifi"
psk="friendPass4567"
}

Python Configuration

The version of Raspberry Pi OS Lite I'm using (May 2020) comes preinstalled with both python 2.7.16 and python 3.7.3.

Raspberry Pi OS Lite doesn't ship with Python's 'pip' installation manager. This can be added using the following commands for both Python 2 and Python 3.

sudo apt-get -y install python-pip
sudo apt-get -y install python3-pip

Because I use Python 3 in my projects, I also like to add an alias to make the python and pip commands use the Python 3 and its Pip.
Edit bashrc file to add the aliases using the nano ~/.bashrc command and add the following:

alias python='sudo python3'
alias pip='sudo pip3'

The bashrc file is loaded each time a terminal (or SSH) is started, but you can reload it using the source ~/.bashrc command.

Serving Framework

Most of the apps I write along with a management view runs on boot using a simple python watchdog script.

The script is setup to automatically launch by editing the /etc/rc.local file and adding the line sudo python3 /share/watchdog.py & before the line exit 0.

As that line shows, I place the watchdog in the /share directory so that I can easily modify it from Windows.

# launched by /etc/rc.local
import os
import time

Processes = [
  { "find": "/share/app1/app.py", "cmd": "sudo python3 /share/app1/app.py", },
  { "find": "/share/app2/app.py", "cmd": "sudo python3 /share/app2/app.py", },
]

while True:
  runningprocesses = os.popen("ps -Af").read()

  for process in Processes:
    if process["find"] not in runningprocesses[:]:
      print("restarting '" + process["cmd"] + "'")
      cmd = (process["cmd"]) + " &"
      os.system(cmd)
    else:
      print(process["cmd"] + " is running.")

  time.sleep(10)

Note that I use "python3" command to ensure that the correct version of python is used for these processes. Care also needs to be used to ensure the libraries are installed for the root user.

Google Cloud SDK

I use Google App Engine for a lot of my web app development. I therefore need to run the Google Cloud SDK. This is done using the following commands (check versions):

wget https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-301.0.0-linux-x86.tar.gz
tar -xzf google-cloud-sdk-301.0.0-linux-x86.tar.gz
./google-cloud-sdk/install.sh
# I allowed it to modify my path (option during install) to make it's functions available to me.

Once setup, you need to login with your Google account using the command gcloud auth login.

Other Software Packages

Install git using the sudo apt-get -y install git command.

Install build-essential package using the sudo apt-get -y install build-essential command.

To install Node and it's package manager (npm) use the following commands (note that the url and paths will need to be checked prior to performing this):

wget https://nodejs.org/dist/v12.18.2/node-v12.18.2-linux-armv7l.tar.gz
tar -xzf node-v12.18.2-linux-armv7l.tar.gz
cd node-v12.18.2-linux-armv7l/
sudo cp -R * /usr/local/
node -v
npm -v

Useful Shell/Terminal Tweaks

Add to end of bash profile using command: nano ~/.bashrc.

System Command Shortcuts

# set chmod 777 recursively from the path provided
alias c777r='sudo chmod -R 777 '
# list directory contents fully and in a vertical list
alias ll='ls -las'
# this is to make the python script (below) runable as a global command.
alias killtxt='python3 /share/killtxt.py'
# make a directory and enter it
mcd () {
  mkdir -p $1
  cd $1
}

# set shell starting directory to '/share'
cd /share

Kill Process By Text Search

I added this python script killtxt.py to my shared directory to match the above alias.

import os, signal, sys
# Ask user for the name of process
name = " ".join(sys.argv[1:])
print("Searching for:", name)
try:
  # iterating through each instance of the process
  for line in os.popen("ps ax | grep '" + name + "' | grep -v grep | grep -v /share/killtxt.py"):
    fields = line.split()
    # extracting Process ID from the output
    pid = fields[0]
    # terminating process
    os.popen("sudo kill -9 " + str(int(pid)))
    # os.kill(int(pid), signal.SIGKILL)
    print("Terminated:"," | ".join(fields))
except:
  print("Error Encountered while running script") 

Python Related Shortcuts

# makes python3.7 deafult and run as root
alias python='sudo /usr/bin/python3'
# makes pip3.7 deafult and run as root
alias pip='sudo /usr/bin/pip3'
# display all python processes currently running
pspy () {
  ps -ef | grep python | grep -v grep | awk '{print $2 "\t" $8 " " $9 " @ " $5 " by " $1}'
}

Google Cloud Shortcuts

# deploy app in current location to GAE with version provided in next argument (eg: 1.0)
alias gdepv='gcloud app deploy -q -v '
# deploy app in current location to GAE with datestamp as version
alias gdep='gcloud app deploy -q -v $(date +"%Y-%m-%d")'
# set current project to the project setup in the alias (I setup an alias for each of my projects)
alias g***='gcloud config set project ***'