Category: System Administration

Simple python web server to demonstrate GET/POST handling and support for embedded Python- round 2

I created a simple python based web server to demonstrate how to use SimpleHTTPServer.SimpleHTTPRequestHandler for handling GET/POST requests. It also demonstrates how to build a system that supports embedded python, templates, create custom URLs, execute local scripts and a number of other things.

It is a single script that acts like a web server, a web server gateway interface and server code so the solution is completely self contained.
Continue reading Simple python web server to demonstrate GET/POST handling and support for embedded Python- round 2

Bash script to install python locally without root permissions

This bash script downloads, builds and installs a particular version of python in the location of your choice without requiring root permissions. It also creates a virtual environment that you can activate to use it. It is licensed under the terms of MIT open source license.

This script is also available on github: https://github.com/jlinoff/pybld.git

Continue reading Bash script to install python locally without root permissions

Bash function that accepts white space arguments

This example shows how to create a bash function that will accept white space arguments. This doesn’t come up often but when it does I have to re-discover how to do it. This little reminder will make that rediscovery process unnecessary.
Continue reading Bash function that accepts white space arguments

p4-whodunit.py – a python script to display who changed each line of code in a perforce depot file

This script (p4-whodunit.py) will analyze a perforce depot file to determine and report who changed each line of code. It is useful for tracking down recent changes that might have caused a problem. It is covered by the MIT license. You can download it here.
Continue reading p4-whodunit.py – a python script to display who changed each line of code in a perforce depot file

Bash script to install gcc-4.9.2 with boost 1.57 on CentOS 5.x, CentOS 6.x and Mac OS X

This week I created a script to install the gcc-4.9.3 compiler and linker with boost 1.5.7. A very simple Makefile is also available. Both files are based on scripts I have created for earlier versions of the compiler. This version also includes tcmalloc for linux platforms.
Continue reading Bash script to install gcc-4.9.2 with boost 1.57 on CentOS 5.x, CentOS 6.x and Mac OS X

Bash script to fix the shellshock vulnerability on Mac OS X 10.9.5 (CVE-2014-6271)

This script will build and install a new version of bash and sh from source that will fix the bash shellshock vulnerability on Max OS X 10.9.5 until the official patch is released from Apple. It requires the XCode command line tools. If you do not have the XCode command line tools installed, I have made pre-built versions of bash and sh available for download.
Continue reading Bash script to fix the shellshock vulnerability on Mac OS X 10.9.5 (CVE-2014-6271)

Bash files to install gcc-4.8.3 and gcc-4.9.1 with boost 1.56 on CentOS 5.x, CentOS 6.x and Mac OS X

This week I created scripts to install gcc-4.8.3 and gcc-4.9.1 with boost 1.56. Here is the simple Makefile.
Continue reading Bash files to install gcc-4.8.3 and gcc-4.9.1 with boost 1.56 on CentOS 5.x, CentOS 6.x and Mac OS X

Bash script to build and install valgrind 3.9.0 with memory limits of 128GB, 256GB and 512GB on linux-x86_64

Valgrind must have fixed memory limits so that it can accurately track the memory allocations of the program under test. On 64 bit architectures it is limited to 64GB. This script will download, modify and build valgrind for larger memory configurations.
Continue reading Bash script to build and install valgrind 3.9.0 with memory limits of 128GB, 256GB and 512GB on linux-x86_64

Bash script to install gcc 4.9.1 and boost 1.55.0 on CentOS 6.x, CentOS 5.x and Mac OS X 10.9

The bld.sh script provided installs gcc 4.9.1 and boost 1.55.0 on CentOS 6.x, CentOS 5.x and Mac OS X for testing. It is another entry in my continuing series of releases for different versions of g++. Hopefully this will make it easier for folks that want to experiment with this version of the GNU C++ compiler.

This script is also available on github: https://github.com/jlinoff/gcc-4.9.1-boost-1.55.

Continue reading Bash script to install gcc 4.9.1 and boost 1.55.0 on CentOS 6.x, CentOS 5.x and Mac OS X 10.9

Bash script to install gcc 4.8.3 and boost 1.55.0 on CentOS 6.x, CentOS 5.x and Mac OS X 10.9

The bld.sh script provided installs gcc 4.8.3 and boost 1.55.0 on CentOS 6.x, CentOS 5.x and Mac OS X for testing. It is another entry in my continuing series of releases for different versions of g++. Hopefully this will make it easier for folks that want to experiment with this version of the GNU C++ compiler.

This script is also available on github: https://github.com/jlinoff/gcc-4.8.3-boost-1.55.

Continue reading Bash script to install gcc 4.8.3 and boost 1.55.0 on CentOS 6.x, CentOS 5.x and Mac OS X 10.9

Automatically install a webserver based on django, nginx and gunicorn on a CentOS 6.5 VPS

This blog presents a bash script that I created to automate the installation of a webserver based on django, nginx and gunicorn on a CentOS 6.5 VPS. The installation installs and configures the necessary system packages, and it installs a number of javascript tools like jquery, jquery-ui, datatables, flot, fancytree and others.
Continue reading Automatically install a webserver based on django, nginx and gunicorn on a CentOS 6.5 VPS

How to support multiple self-contained production quality web sites based on django, postgres and nginx on a single server using mkwebsite

The mkwebsite package allows you to install multiple completely self contained production quality websites on a single server for development and debugging without root privileges. I have used it for web development on linux and Mac OS X. The sites generated can also be used in production.

I developed it because I wanted to be able to completely remove my development sites without leaving any system wide packages around that I didn’t need and because I wanted to compare security settings side by side for sites that were otherwise identical.

It is very easy to use, simply download the package, extract it and run the associated install script for each website that you want to create. It takes care of all the system administration details (like database and server configuration) so that you can concentrate on development.
Continue reading How to support multiple self-contained production quality web sites based on django, postgres and nginx on a single server using mkwebsite

Password Selection Strategies

Choosing secure passwords is a tricky business. This document describes some strategies for creating and selecting them to effectively protect your accounts. The key points are that you need long passwords that are hard to crack but easy to remember, that you should have different passwords for different accounts and that for critical accounts you should have different usernames as well.
Continue reading Password Selection Strategies

How to add a superuser in Django 1.6 from the command line

This python script shows how to add a super user and display all super users in your Django 1.6 installation from the command line. I wrote it to support automated configuration of my django installations.
Continue reading How to add a superuser in Django 1.6 from the command line

Bash script to install gcc 4.8.2 and boost 1.55.0 on CentOS 6.4, CentOS 5.5 and Mac OS X 10.9

I recently installed gcc 4.8.2 and boost 1.55.0 on CentOS 6.4 to take advantage of the improved support for C++-11. It was painful to do manually so I automated the process with this script: http://projects.joelinoff.com/gcc-4.8.2/bld.sh. Hopefully this will make it easier for folks that want to experiment with the latest version of the GNU C++ compiler.

If you are interested in a more recent version, I recently added a post that provides a script to install gcc-4.8.3 and boost 1.55.0.

Continue reading Bash script to install gcc 4.8.2 and boost 1.55.0 on CentOS 6.4, CentOS 5.5 and Mac OS X 10.9

Create and use a local mirrored yum repository for CentOS 6.4 x86_64

A local mirrored yum repository is a copy of the standard yum repositories for CentOS that exists on your local network. It is kept up to date by synchronizing to an external site. If you have more than a few local hosts, using a local mirrored repo can significantly reduce internet traffic and speed up the update process.

This blog describes how to create, sync, test and use a local mirrored repo for CentOS 6.4. You will be shown how to create the repo, keep it in sync using cron, test the implementation and how to update the /etc/yum.repos.d directory on each host. It also provides 4 bash scripts that you can customize for your site to make maintenance easier.
Continue reading Create and use a local mirrored yum repository for CentOS 6.4 x86_64

Count all Confluence pages in python using XML-RPC API

I was recently asked how to count the total pages on a 4.2 Confluence server so I provided this python script that shows how to use the XML-RPC API to do it. The technique can be used for more than counting pages. The API provides all sorts of useful operations, like adding users. For more information on the available methods see this page: https://developer.atlassian.com/display/CONFDEV/Remote+Confluence+Methods.
Continue reading Count all Confluence pages in python using XML-RPC API

Remote command execution in python using paramiko that supports arbitrary input

I recently decided to use paramiko to develop a remote command execution tool.

It was very easy to setup initially and ran much faster than my existing pexpect implementation but it had a problem with sudo commands because they required the password to be provided as input.

I solved the problem by using a pseudo-terminal and by creating my own ChannelFile objects for stdin and stdout/stderr. The solution should be general enough to handle any case that requires simple input but it is not as flexible as pexpect. I hope that you find it useful.
Continue reading Remote command execution in python using paramiko that supports arbitrary input