Fortune Cookie Says

Tomorrow your friend or partner will tell you some exciting news.

Fortune Cookie Says

You will soon be crossing the great waters.

Project Backup Script

I am using trac and subversion on my current project. I wrote the following script to do a full backup of trac and subversion and burn the results to a CD.

#!/bin/bash

## Backup subversion and trac and burn the backups to a CD.
##

## applications
##
TRACADMIN=/usr/bin/trac-admin
SVNADMIN=/usr/bin/svnadmin
MKISOFS=/usr/bin/mkisofs
CDRECORD=/usr/bin/cdrecord
DATE=/bin/date

## directory and file paths
##
REPO=/opt/repo/tde
TRACPROJ=/opt/trac
BACKUPDIR=/home/steve/backup
TMPDIR=$$
CDDEVICE=/dev/cdrom
SVNDUMPFILE=$BACKUPDIR/$TMPDIR/temp/svn_dump.txt
DATEFILE=$BACKUPDIR/$TMPDIR/temp/backupdate.txt
LOGFILE=$BACKUPDIR/$TMPDIR/backup.log

if [[ $EUID -ne 0 ]]; then
 echo "This script must be run as root"
 exit 1
fi

## check for the directory structure and attempt to create if needed
##
if [[ ! -e $BACKUPDIR ]]; then
 if ! mkdir $BACKUPDIR; then
     exit 1;
 fi
fi

if [[ -e $BACKUPDIR/$TMPDIR ]]; then
 exit 1;  # temp dir already exists
else
 if ! mkdir $BACKUPDIR/$TMPDIR; then
     exit 1
 fi
fi

if ! mkdir $BACKUPDIR/$TMPDIR/iso; then
 echo "Failed to create $BACKUPDIR$TMPDIR/iso" >> $LOGFILE;
 exit 1;
fi

if ! mkdir $BACKUPDIR/$TMPDIR/temp; then
 echo "Failed to create $BACKUPDIR/$TMPDIR/temp" >> $LOGFILE;
 exit 1;
fi

## dump subversionto SVNDUMPFILE
##
echo "Dumping subversion" >>$LOGFILE
echo "" >>$LOGFILE
if ! $SVNADMIN dump $REPO > $SVNDUMPFILE 2>>$LOGFILE; then
 exit 1;
fi

## hot backup trac
##
echo "" >>$LOGFILE
echo "Performing hotcopy of trac" >> $LOGFILE
echo "" >>$LOGFILE
if ! $TRACADMIN $TRACPROJ hotcopy $BACKUPDIR/$TMPDIR/temp/trac \
         1>>$LOGFILE 2>>$LOGFILE; then
 exit 1;
fi

## create a file with the date time the backup was done
##
echo `$DATE` > $DATEFILE

## create iso file
## mkisofs does not seem to return a status value so we dont know if it
## actually succeeded.
##
echo "" >>$LOGFILE
echo "Making ISO file" >> $LOGFILE
echo "" >>$LOGFILE
$MKISOFS -fRrlJ -o $BACKUPDIR/$TMPDIR/iso/backup.iso \
     $BACKUPDIR/$TMPDIR/temp 1>>$LOGFILE 2>>$LOGFILE

## burn the iso file to a cd
## cdrecord does not seem to return a status value so we dont know if it
## actually succeeded.
##
echo "" >>$LOGFILE
echo "Burning the CD" >> $LOGFILE
echo "" >>$LOGFILE
$CDRECORD -eject speed=24 dev=$CDDEVICE $BACKUPDIR/$TMPDIR/iso/backup.iso \
         1>>$LOGFILE 2>>$LOGFILE
#if $CDRECORD speed=24 dev=$CDDEVICE $BACKUPDIR/$TMPDIR/iso/backup.iso \
#            1>>$LOGFILE 2>>$LOGFILE; then
#    exit 1;
#fi

## clean up if everything was OK
## since we don't know if mkisofs or cdrecord succeeded do not
## automatically delete the director.
##
#rm -rf $BACKUPDIR/$TMPDIR

The only thing in this script to make note of is the line TMPDIR=$$. The TMPDIR variable is will be the name of the temporary directory to backup to and this is set to the PID of the process which is returned by $$.

To automate the backup I run the script through cron. To setup the cron entry I ran the following commands as root.

[root@myhost]# export EDITOR=vi
[root@myhost]# crontab -e

This will bring up a vi editor with root's crontab file. I then added the following line and saved the file.

59 23 * * 5 /home/steve/bin/sp_backup.sh

The script will now be run every Friday at 11:59 PM. The only problems that I encountered with this script is that neither mkisofs or cdrecord seem to return the UNIX standard value of 0 for success and non-zero for failure. I was going to have the script delete all of the directories and files that were created. But because I can't determine if the script was successful I decided to leave the files in place and to manually ensure that the backup was successful.

Fortune Cookie Says

Maybe you can live on the moon in next century.

Processing Apache Access Logs

I need to keep track of who is logging in and using our project's issue tracking and wiki sites. I wanted a simple list of who has logged in on what day. I came up with the following command line to give me what I wanted.

cat access_log | awk '{ if ( $3 !~/-/ ) \
{print $3" " substr($4,2, index($4,":")-2 )} }' \
| sort | uniq

This gives me a list of users and login dates grouped by user as follows.

bambam 25/Aug/2008
barney 25/Aug/2008
barney 26/Aug/2008
fred 25/Aug/2008
fred 26/Aug/2008
fred 27/Aug/2008

Fortune Cookie Says

Endurance and persistence will be rewarded.

CDMA SMS User Data

In this post I will describe the decoding of the user data portion of the bearer data field of a CDMA SMS message. See my first post for the structure of the bearer data field.

The User Data subparameter is the portion of the Bearer Data field of a CDMA SMS message that contains the actual message or payload. The user data is made up of an integral number of octets and is 0 padded as needed. The user data subparameter is documented in section 4.5.2 of the IS-637 spec. A PDF of this spec is available on the 3GPP2's web site. A PDF of the spec is also available on the TIA's website. The structure of the user data subparameter is:

• The subparameter ID of 8 bits which is the constant that identifies the start of this subparameter in the bearer data.
• The subparamter Len of 8 bits which is the number of octets that make up the value portion of this subparameter.

These two portions of the user data are processed as part of the bearer data. The rest of the user data subparameter is processed separately. The remaining fields of the user data are:

• The message encoding is a 5 bit value that indicates which encoding scheme was used for the message.
• The message type is an optional 8 bit value that is used only if the encoding is an IS-91 extended protocol message. See the specification document for the details.
• The num fields is the number of data elements, of the size specified by the encoding and message type, that the message contains.
• The chari portion contains the actual text or payload of the message.
• The final portion the message is 0-7 bits of 0 padding as needed to fill the last octet.

The first step I took in decoding the user data was to write a function that determines the message encoding, the size of the data elements, and the starting byte of the message. To this I first start by defining some constants.

// masks and values for processing the user data fields
//
#define ENCODING_MASK   0xF8
#define ENCODE_OCTET    0X00
#define ENCODE_IS41     0X08
#define ENCODE_7BIT     0X10
#define ENCODE_IA5      0X18

#define MST_BYTE_1_MASK 0xE0
#define MST_BYTE_2_MASK 0x1F
#define NF_BYTE_1_MASK  0xE0
#define NF_BYTE_2_MASK  0x1F

// standard sized type definitions
//
typedef char                sint_8;
typedef short               sint_16;
typedef int                 sint_32;
typedef long long           sint_64;

typedef unsigned char       uint_8;
typedef unsigned short      uint_16;
typedef unsigned int        uint_32;
typedef unsigned long long  uint_64;

Then I write the routine to do the initial processing. This function is writtento call another function that is will handle the actual decoding of the message.

void
decode_user_data( uint_8 *userData, size_t sz )
{
  uint_8 *ud  = userData;             // current element
  uint_8 *lud = userData + sz - 1;    // last element

  uint_8  encoding;
  uint_8  mst;
  uint_8  numFields;
  uint_8 *nextByte = ud + 1;

  int i;
  for( i = 0; i < sz; i++ ) {
      printf("%X\n",userData[i]);
  }

  mst = 7;
  encoding = *ud & ENCODING_MASK;
  switch( encoding ) {
      case ENCODE_OCTET:
          mst = 8;
          break;

      case ENCODE_IS41:
          mst = (( *ud       << 5 ) & MST_BYTE_1_MASK ) +
                (( *nextByte >> 3 ) & MST_BYTE_2_MASK );
          ud++;
          nextByte++;
          break;

      case ENCODE_7BIT:
      case ENCODE_IA5:
          break;

      default:
          perror( "unknown paramters\n");
          exit(0);
  }

  numFields = (( *ud       << 5 ) & NF_BYTE_1_MASK ) +
              (( *nextByte >> 3 ) & NF_BYTE_2_MASK );

  printf("numFields: %d\n", numFields );
  printf("first byte: %X\n", *nextByte);

  switch( encoding ) {
      case ENCODE_7BIT: {
              char *text = decode_7bit_ascii(
        nextByte, numFields, 3 );
              printf("The text message is: '%s'\n", text );
              free( text );
              break;
          }

      case ENCODE_OCTET:
      case ENCODE_IS41:
      case ENCODE_IA5:
          perror( "requested encoding is not implmented\n");
          return;
          break;
  }
}

The only message type that I am concerned about with this is the 7bit packed ASCII. I will show how to unpack this into a NULL terminated string in another post.