Tag Archives: MySQL

MySQL Connector and Web-Interface for DX Spider

DXSpider is Copyright © 1998-2007 by Dirk Koopman G1TLH. http://www.dxcluster.org

Purpose:

The propose of the script is to extract the DXSpots from the DXSpider data files and write them into a MySQL database. This gives the facility of interfacing to the spots in many ways, mainly from a web interface, which was the main reason for developing this script.

MySQL offers a fast and query based database which can also be accessing from log book programs ect.

Prerequisites:

  • Linux Operating System (Will probably work on Windows also)
  • DXSpider
  • Apache 2.2
  • MySQL 5
  • PHP5

Create the database schema:

mysql> CREATE DATABASE `dxcluster` ;

Create the tables:

CREATE TABLE `dxcluster`.`spots` (

`freq` DOUBLE( 12, 3 ) UNSIGNED NOT NULL ,
`dxcall` VARCHAR( 85 ) NOT NULL ,
`datetime` VARCHAR( 85 ) NOT NULL ,
`commets` VARCHAR( 100 ) NULL ,
`spotter` VARCHAR( 85 ) NOT NULL

) ENGINE = MYISAM

Now we create the scripts for datacollection:

Create a file called start.txt and newspots.dat

touch start.txt

touch newspots.dat

Now create a file called dxmysql.sh and insert the following:

i=$(cat /spider/data/spots/2008/start.txt)

doy=`date +%j`

v=`cat /spider/data/spots/2008/$doy.dat |wc -l`

sed -n "$i,$v p" /spider/data/spots/2008/$doy.dat > /spider/data/spots/2008/newspots.dat

let "v++"

echo $v > /spider/data/spots/2008/start.txt

mysql -u <mysqlusername> -p<mysqlpassword> -e "load data infile '/spider/data/spots/2008/newspots.dat' into table dxcluster.spots fields terminated by '^' lines terminated by '\n';"

Now create a cron job (/etc/crontab) as follows:

0 0 * * * root echo "1" > /spider/data/spots/2008/start.txt

* * * * * root /spider/data/spots/2008/dxmysql.sh > /dev/null

Please ensure that the DXSpider server is set to use UTC as timezone.

Upload the PHP scripts to your /var/www directory

You only need to edit the file clusterdb.inc.php to set your MySQL user and pass.

Now access http://your_site/cluster.php

This software is provided as-is and comes with no guarantees. It is provided under the Creative Commons licence. Please visit http://creativecommons.org/licenses/GPL/2.0/ for more details.

Please leave the “Powered by 9H1LO DX Engine” and link to www.9h1lo.net on the html pages.

New update for 2010. The dxmysql.sh script has been updated to support auto year change:

#!/bin/bash
year=`date +%Y`
i=$(cat /spider/data/spots/$year/start.txt)
doy=`date +%j`
v=`cat /spider/data/spots/$year/$doy.dat |wc -l`
sed -n "$i,$v p" /spider/data/spots/$year/$doy.dat > /spider/data/spots/$year/newspots.dat
let "v++"
echo $v > /spider/data/spots/$year/start.txt
mysql -u <USER> -p<PASS> -h <HOST> -e "load data local infile '/spider/data/spots/$year/newspots.dat' into table dxcluster.spots fields terminated by '^' lines terminated by '\n';"