Linux/Unix host
Linux/Unix destination with SSH server

Ready, Set, Go!

Getting right down to it, here is a quick simple snippet I use quite often to transfer batches of files (often text, or other highly compress-able files) :

tar cz file.foo folder foo/* | ssh user@destination.host.com ‘cat > ~/file.tar.gz’

Today I found myself wanting to transfer an entire CD to a remote host, normally I’d just find a place to download it, but being a proprietary disc, that wasn’t an option.  This was too easy, but keep in mind if you don’t have a good upload speed, this can take awhile:

dd if=/dev/cdrom | ssh user@destination.host.com ‘cat > cd.iso’

Now of course you can mount it on the remote server:

mount /home/user/cd.iso -o loop /mount/path

Any questions?

This article assumes you’re already familiar with Mercurial, if you’re not, there are plenty of places to find a quick and basic tutorial.  Chances are if you’ve ever used version control, you’ll be able to follow.  Some of the Mercurial commands used don’t show up until later versions of Mercurial (1.6), so if you’re running an old version (shoutout to all those Lenny users), there are ways to accomplish the same, but the syntax and ease-of-use is a little more difficult.

What is the patch queue?

Mercurial’s patch queue is probably seen differently depending on the environment it’s used in.  Personally, I like to think of it as “flexible changesets”, meaning until I’ve finalized my queue and pushed it upstream I can work on any revision in the queue and “refresh” the revision to include changes I make.

What does that mean?

If you’re familiar with version control, you know that generally speaking, changesets are immutable, if you wanted to add some extra comments to a file, you’d have to make them in a new revision, separate from the changes that make the comments relevant.  I like to keep my version control history clean and concise, and that is where the patch queue really shines.

The patch queue in the workplace

I work with many various Mercurial repositories on a daily basis and can have projects in-progress across multiple repositories.  Normally, I’m not going to finish a project in a single sitting, so the very first thing I do when I start working is create a patch, named for set of changes I’m planning to implement, but the name doesn’t really matter.

Example: hg qnew patch_queue_example

This initializes the patch queue and puts the “patch_queue_example” patch on top of the stack.  I am now free to make changes, and can then save them by “refreshing” the patch I’m in.

Example:
touch newfile
hg add newfile
echo “Test” >> newfile
hg qrefresh

Now, if you look at the head of the repository (hg heads), you’ll see a new changeset, with the patch name.

changeset:   0:6627f1eb79a5
tag:         patch_queue_example
tag:         qbase
tag:         qtip
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:14:59 2011 -0400
summary:     [mq]: patch_queue_example

This changeset includes my changes to “newfile”, which you can see by running “hg export tip”

Now, that I’ve made those changes, I want to work on changes to the same repository, but they don’t relate to my previous changeset, so I create a new patch.

hg qnew non_related_changes

Looking at “hg heads” again, I see a new revision:

changeset:   1:52cfaec4afa1
tag:         non_related_changes
tag:         qtip
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:19:38 2011 -0400
summary:     [mq]: non_related_changes

Now, I’ll go about making new changes:

echo “These changes are unrelated to my previous changes, so I’ve created a new patch” > unrelated_changes
hg add unrelated_changes
hg qrefresh (again, this is rolls the current changes into the current patch)
hg heads

You’ll notice the short revision id has stayed the same (1), but the node has changed

changeset:   1:52cfaec4afa1
tag:         non_related_changes
tag:         qtip
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:19:38 2011 -0400
summary:     [mq]: non_related_changes

You can see the changes by running “hg export tip”, you’ll see the “unrelated_changes” with the text we’ve echoed into it.

Great, get to the useful stuff

Now, let’s say I wanted to go back and edit the “patch_queue_example” changes, I decided I wanted to add some more files in that changeset.  Normally, you’d just add the files, and make a new commit, providing a commit message stating you forgot to do something in the previous commit, or something similar.
With the patch queue I can simple “pop-off” the current changeset and go back to editing “patch_queue_example”

hg qpop

popping non_related_changes
now at: patch_queue_example

So we’ll add a few files

echo “foo” > example1
echo “bar” > example2
echo “baz” > example3

Now, add them to Mercurial

hg add

adding example1
adding example2
adding example3

Looking at a “hg status”, you’ll see the files have been added:

A example1
A example2
A example3

Now, let’s save the changes and go back to the “unrelated_changes” patch

hg qrefresh
hg status

You’ll notice the files are no longer listed as added, because the changes have been “commited” ( != hg commit).

Let’s move back to the “unrelated_changes”, by “pushing” it back onto the top of the stack

hg qpush

applying non_related_changes
now at: non_related_changes

At this point, I hope you see where this is going, you can continue doing this until you’ve finished all the relevant work.
Now it’s time to convert them into “real changesets” so they can be pushed upstream, which I haven’t mentioned until now, you don’t want to try pushing patches upstream, namely because they should be considered unfinished until you’ve converted them, but also because each “hg qrefresh” you do, the node will be updated (Mercurial generates the node id based on changes/history, it will change if you change anything).

To convert these into real changes I want to do two things
1) Edit the commit messages
2) Remove them from the patch queue

Editing commit messages:
hg qpop (to move back to the first patch)
hg qrefresh -e (should drop you into a text editor, whatever you put here is the commit message)
hg qpush (to move to the second patch)
hg qrefresh -e (edit second patch commit message)

Removing from the patch queue:
Note: You can remove them one at a time using ‘hg qfinish $rev’, but we’re going to do it all at once.
hg qfinish -a

That’s it!  Now take a look at your log:

hg log

changeset:   1:1bb3052c69fc
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:42:47 2011 -0400
summary:     This was the second commit in the Mercurial patch queue example

changeset:   0:5c616b8e4745
user:        Tim <example@example.com>
date:        Wed Oct 12 09:41:56 2011 -0400
summary:     This was the first patch in the Mercurial patch queue examples

These changes are ready to be pushed upstream.

This article of course does not cover extensive use and some of the neat things you can do while working in the patch queue, for the brave, here are some other commands I use frequently, maybe you can figure out where they’re good for.

Extra commands:
hg qnew -f new_patch_name
hg qpush –move patch_name

1) SSH Tunnels – Each and every time I’ve used this to cut half the time off a task I remember just how great SSH is.

When to use: Anytime you need internal access to a remote network. Or perhaps your company filters your internet usage and you need to check your gmail, this allows you to TUNNEL to your internet back at the house and ask it to do all the talking for you.

Example: I’m working from home, but I really need to access a ‘local’ resource on the remote network, it could be something as simple as connecting to the router on-site that doesn’t have remote access setup.

Howto Linux: If you’re a linux user, use the “-D” option with your standard ssh command. IE: ssh -l username -D 54321 sub.domain.com

Howto Windows: Get PuTTY. You’ll likely want to save this configuration so, open PuTTY, type in the hostname, give the session a name, on the left hand panel navigate to Connection->SSH->Tunnels. Under “Add New Forwarded Port” pick a port number, I like to use something high in the range, 54321 or you can pick your own. There are two rows of radio buttons, on the top choose “Dynamic” and leave the other as “Auto”, click “Add”. Now before connecting, on the left hand panel goto Sessions and click “Save”, now click “Open”

Utilizing the tunnel: What this essentially does is creates a secure SOCKS proxy for you to use. In Firefox, goto Preferences->Advanced->Network->Settings and set the SOCKS proxy address to “127.0.0.1″, that’s right, localhost (but don’t use ‘localhost’), then enter the port you chose earlier. Click “Ok” and you’re now surfing securely through SSH.

2) Converting movies to pocket size (linux only):
I have a simple Creative Zen 8GB Mp3 Player, it was cheap and works exactly as expected. It plays video of the same quality popular cell phones use, occasionally I want to watch a TV episode so I pieced together a simple little alias to make it quick and painless, you’ll need mencoder for this, if your on debian/ubuntu it’s as simple as ‘apt-get install mencoder’

Code (I put in ~/.bashrc): alias encode=’mencoder -ovc xvid -oac mp3lame -vf scale=320:240 -xvidencopts fixed_quant=4 -o ${args[0]} ${args[1]}’

Usage: encode outputFile.avi /path/to/file/to/convert.flv|avi|mpg

3) Using axel for super fast downloads
Situation: You’re remotely connected to a server you adminster, you’re crunched for time and need to download a file, maybe an ISO or perhaps a service pack, but you need it fast. You have no problem finding links to get it, but which do you use, which will be faster? The solution, grab 3 or 4 of those links and use axel.

Usage: axel -n X url1 url2 url3…
Replacing ‘X’ with the number of connections you want to use, passing as many URLs in as you’d like. In most cases, 3 or 4 seem to be enough to max out my 2MB/s.

Mercurial is a cross-platform, distributed revision control tool for software developers.

Easy way to view most recent changes:
Command: hg export tip | colordiff | less -R

Useful for: Quick code reviews, or just to peek at what you’ve recently done

Manipulating text files:

SED: (stream editor) is a Unix utility that (a) parses text files and (b) implements a programming language which can apply textual transformations to such files.

Command: sed ‘s/\t/”,”/g;s/^/”/;s/$/”/;s/\n//g’ > filename.csv
In action: mysql -uuser -ppass DBNAME -e “SELECT foo FROM bar WHERE baz = ‘world’” | sed .. > output.csv

Explanation:

• “s/\t/”,”/g;” – Search for tabs, replace with commas
• “s/$/”/;” – Replace last of line with quote
• “s/\n//g” – Remove newlines

Useful for: This is specifically used for converting MySQL results to csv. I build lots of mailing lists for my employer.

AWK: is a programming language that is designed for processing text-based data, either in files or data streams, and was created at Bell Labs in the 1970s

Command: awk ‘BEGIN {OFS=”|” FS=”|”}; {print $1, $3, $5}’ file.csv
Explanation:

• OFS – Output field separator
• FS – Field separator
• Printing fields 1, 3, and 5

Useful For: Pulling text out of delimited list and re-organizing the data, quickly and gui-free

WARNING: If you choose to do this, that’s your choice, read this through before even thinking of diving in. In other words, I am not responsible for ANYTHING that happens as a result of what YOU choose to do. I also assume your familiar with linux and the distro of your choosing, really, I won’t be explaining some things as this is already going to be lengthy.

Note: This may not be the most secure way to do things, but I feel it’s stronger than windows, I have better backups (flyback for gnome, and a few scripts I’ve written), and no one has “Administrator” access, except me of course.

For the past 2 years or so I’ve been the ‘Tech Guy’ at two vet offices in my “free” time.  I’ve had one Windows 2000 server, which was actually the most reliable, compared to the Windows Small Business Server 2003.  I’ve experienced more problems than I care to recall, let’s suffice it to say, the workstations in the larger office were unreliable on network, even after a network rebuild (replace and tested all cable, made sure machines were clean, running the server on a gigabit network connection, all fun stuff).  Finally it got to the point where I was spending all my free time there, no good.  I am far more comfortable on any linux box than a windows machine, so I decided to take the plunge and make it work, installing linux servers that is. It may sound easy, but even since the switch I’ve had many unexpected surprises.  

Office 1 was something like (It has come a LONG way):

• Windows 2000 server, static IP, gigabit network, file and print sharing on, rdp, that’s about it
• 15 Workstations – Slight variations of course, 2 or 3 year old dell desktops, 2.4 Ghz, 256-512mb ram, wired network
• Most cable all came back to a 24 port switch, old and hot, however they had expanded since then so they installed 2 ROUTERS in certain points in the office, when I first got there they were both broadcasting wireless and DHCP, in addition to:
• Bellsouth/Westel DSL Modem/Router – oh JOY! (Broadcasting DHCP also)

Needless to say this office was unreliable at best.  It’s taken some time and effort but now:

• Server (AMD 3.2Ghz, 4 gig ram, locking case, enough storage, headless now, after the switch from windows)
• I bridged the Westel to a WRT54GS running DD-WRT v24, replacing a simple 4 port linksys switch (forgot to mention that)
• Installed a new 24 port switch, 2 gigabit ports, with a fan :-p
• Obviously turned the routers in the building to switches
• All cable tests good, removed anything sketchy

 
After installing Windows Server 2003 and still running like crap, I can’t diagnose anything because either I’m just not that Microsoft certified, or any decent software costs too much for me to justify, anything more than 10 bucks or so.  So what do I do?  Rip that Windows COA off the case, throw in my Ubuntu Intrepid cd I’m on my way (actually I’d been thinking/tinkering with this for months).  Oh, a few things that might be problems:

• Avimark doesn’t support linux.  Scares the hell out of the Vet.
• Applying Avimark updates, online updates work just fine
• Running reports, they ‘need’ to be run on the server
• Vet ‘Admin’ people need access to Avimark on the server, more than just for reports.
• You won’t/shouldn’t run the Avimark “Server” edition on linux
• Sharing remote (over the internet) printers via CUPS

The list goes on, but let’s go over a few benefits:

• Remote admin is so much simpler
• You get a real server (Apache, SSH – complete with tunneling to easily bring their network to your home network, CUPS)
• Users won’t have physical access to your server
• Nagios, useful for server and workstation failures
• Crons (Backups, notifications, etc.)
• OpenVPN
• I also setup Wiki and put useful information for the employees there (IE: Installing a printer…)
• Open source help desk of your choosing
• Oh yea, SAMBA, the single most important tool for replacing windows servers.

Again, list goes on but I digress, let’s begin.

You can pretty much follow http://www.howtoforge.com/perfect-server-ubuntu-9.04 if you’re reading this I trust you can do your own install and I don’t need to tell you much, you’ll need samba, I prefer to have Apache, mySQL, SSH, and it’s very handy if you can send mail, and we’ll be using VirtualBox 2.0.4, not OSE, it’s easier.

So once your all done and booted up, sign in, check the internet, etc.  Let’s get to the gritty, config!

I’ve started with the default smb.conf, this is along the lines of what I have now:
View it here 

I’ve found it very important to make sure Avimark is happy with the permissions, access a good portion of the features before you give up. It’s absolutely necessary to make sure the samba user has access to all files that clients will create, as well as files that will be created when “Vet Admin” are using the VRDP server, I had fun with that for some time.

Now, you’ll need a direct connection to the server to do things like running reports and rebuilding indexes, that’s where VirtualBox comes in.

Modify /etc/apt/sources.list

Add: deb http://download.virtualbox.org/virtualbox/debian intrepid non-free
Run: wget http://download.virtualbox.org/virtualbox/debian/sun_vbox.asc -O- | sudo apt-key add -

Then:
sudo aptitude install virtualbox-2.2 (or newest version)

Now, make sure your user is allowed to use VirtualBox

sudo adduser `whoami` vboxusers

Create the machine and the disk, setup the vrdp while we’re here:

VBoxManage createvm foo (be sure to add quotes if you use spaces)
VBoxManage createhd -filename .VirtualBox/foo.vdi -register -size 5120
VBoxManage modifyvm foo -hda "/home/ttpears/.VirtualBox/foo.vdi"
VBoxManage registerimage dvd /path/to/your/legal/copy/of/winxp/PRO.iso
VBoxManage registerimage dvd /usr/share/virtualbox/VBoxGuestAdditions.iso
VBoxManage modifyvm foo -dvd /path/to/your/legal/copy/of/winxp/PRO.iso
VBoxManage modifyvm foo -vrdp
VBoxManage modifyvm foo -vrdpmulticonn 1
VBoxManage startvm foo -type vrdp (I've turned this into an alias)

Using either rdesktop or mstsc connection to the host ip address, you should see the machine booting up. Do the windows install, then change the cd (we added the GuestAdditions cd to VirtualBox earlier), shut down the vm:

VBoxManage controlvm foo poweroff
VBoxManage modifyvm foo -dvd /usr/share/virtualbox/VBoxGuestAdditions.iso
VBoxManage sharedfolder add foo -name "sharedfolder" -hostpath "/srv/avimark"
VBoxManage startvm foo -type vrdp

Now, connect again and install the utilities, makes the whole experience much more pleasing.
Open a run prompt and type ‘net use Z: \\vboxsvr\sharedfolder’
If you know anything about Avimark, you know they don’t condone using mapped drives, but they only do it because it’s unreliable, this is a whole different story here, as we are physically on the same computer.
Open up the Z:\, navigate to Avimark.exe and create the shortcut on the desktop.
You’ll probably want to print from here too, you can browse the network by ip, open a run box and type ‘\\192.168.XX.XXX’, Open or connect the printer of your choice and set the printer in Avimark. I make RDP shortcuts on the client workstations that will be using the server connection, now you might have noticed we enabled multiconn, this allows more than one user, but not in the traditional RDP way, more VNC like, all users control the same screen.

Oh I wish it ended there, but one of the vet admins had been using a RDP from Office 2–> Office 1 and printing at Office 2, ….REALLY? Well, cups to the rescue, install and setup cups, you’ll need port 631 for this (I mentioned I wouldn’t explain everything, right?). Now you can do this at both ends and make it two way, or you can just choose one office to print remotely. So connect to the other office’s VRDP connection, add a Internet Printer (IPP), use http://foo.fqdn.com:631/printers/printername, set as the default and set it in Avimark.

Again not explaining much except ‘how it’s done’, on top of that I broke the money rule, I broke the rule, I wanted a rock solid remote admin for the workstations so I went with RAdmin, installed the server and a matching user name and password, set them to incrementing port numbers, forwarded the ports, saved all the connections in a folder, very nice, if you need to copy the settings you can find the file in Application Data. I find it’s very fast, and since some of the workstations have XP home, RDP wasn’t an option, DSL isn’t very friendly to VNC.

What more can you want? Oh, a monitoring service, now this may be a little overkill for some people, but I like to know what’s going on at a glance, not to mention my vet doesn’t hesitate to blow up my phone. I installed Nagios, I monitor Avimark.exe, and an assortment of system stats on each workstation, keeps me on my toes.

Again, a disclaimer, don’t do this on your production machine, set it up in a dev environment and just switch it over on a weekend.

I signed up a few days ago, got the welcome email, and since I use zoneedit.com to manage the DNS (we’ll get to this later) I was in no hurry to get the DNS moved over.  I check my email last night and find an email from the hosting company saying I may be in violation of the TOS!  What?  I’m thinking I don’t even have a website there yet since I hadn’t switched the DNS, so I give them a call (mind you, they’re available 24/7 by phone via 1-800, very cool).  A guy picks up after waiting only a moment or so, again very cool, I explain the situation and ask how I could possibly be breaking the TOS before even have a website there.  The guy on the phone takes a moment to look up everything and understand what’s going on, they’ve flag it as a hacker web site.  “Really…. why?” I ask.  Of course at this point in my head I repeat the domain name, OH!  I think I understand now, so I suggest the guy on the phone visits the site to have a look for himself, which he directs to someone who can actually make the call.  We make small talk on the phone while we wait for the “reviewers”, the guy is very nice and rather knowledgeable, again very cool, he actually visits the site himself while we’re waiting and recognizes asterisk and x10!  After a few minutes the reviewers decided it was okay and instantly my account was reinstated, sweet..  This whole host switch was motivated by the terrible support and poor quality of totalchoicehosting.com, which I had been with for quite a few years, while other hosts improved over the years my host added nothing and kept the prices the same.  Just a small comparision:

TotalChoice: 2.4 gigs space
80 gb a month transfer
no ssh
5$ a month

Gatorhost: 600gb
6000 gb a month transfer
SSH!
8$ a month

So even if I would have bumped up to the 8.95$ plan with totalchoice the space went up to 3gb and transfer to 140gb a month, and still no ssh, still way less than I’m getting and already and
the support with hostgator has been great!

Now to get back to zoneedit.com.  This company has been around for quite some time now, I think I’ve been with them at least 8 or 9 years now, for free.  They offer a DNS service that rivals anything you’ve ever used.  The site may not look like much, but they let you manage up to 5 free “zones”, or domains.  So when you buy a new domain name, you use the nameservers they give you and they let you manage your dns settings, but heres the catch, it’s nearly INSTANT, anyone who ever has changed host knows that in the 24-72 hours after changing your website can be all out of sorts.  Not only do your changes take effect very quickly, you can add sub-domains that point to different ips, or even point something like foo.hackyourworld.com to google.com without the viewer even seeing the google.com.  If you find yourself wanting more control over your domain name, this is the ticket!

With all the changes I am planning on making this more regular, topics for the future will likely include bash, testing web pages, using ssh for demonstrations and opening your garage door by bluetooth proximity.  Hope you enjoy.

* Looking at the stats from the traffic of this site we’re going to assume everyone already uses Firefox.

SSH Secure Shell Client – Assuming you have a linux computer, or anything you use SSH for, this has got to be one of my most used tools, sure there is putty, but it just doesn’t provide the full functionality that this discontinued free tool does, and still is lightweight. Includes built-in support for SFTP transfers, saves profiles, supports tunneling.

Media Monkey – I finally kicked winamp to curb after I found this little gem. Keeps my music in a fashion that I can quickly find what I’m looking, or just double click the auto-generated favorites list to hear songs I commonly search for, it does this without eating all my memory or processor, something winamp hasn’t been able to do in a long time. I can also keep my desktop shared music in the library and remove it from the playlist when I’m away from my home with just a few mouse clicks, or it just skips it without fussing.

µTorrent – Who doesn’t use this now days? The lightest client, but still has everything I need. I always seem to get the best speeds using this, it could be that Azureus just eats up so much memory that it can’t keep up with the 1200KB I get off Tvtorrents.com . Make sure to tweak your connection settings to handle more connections than the default, assuming you have a fast connection. If your using XP you’ll probably want to increase the amount of half-open connections you can have (default is 10), visit this site and download the tool.

LeechFTP – If anyone used this FTP client before you probably still do, again I’m always looking for the lightest client, efficient, and to the point. It’s from 1996 I think, but since I’ve started using it, I’ve tried just about every one out there and I can’t seem to find one that is as light and works as easily. Supports bookmarking, ASCII file transfers, and multiple threads. Was last updated in the very late 90′s, maybe one day we’ll get an update, but it still works just as well as it always has

VLC Media Player – Does it get any better than this player? I think not. It will play damn near anything you throw at it, including unfinished AVI videos and BIN/CUE. Supports playlist, not the best interface for the playlist, but effective and supports drag-n-drop. You can skin it if you’d like, but I find it works fine out of the box, and I usually full screen everything anyway. Did I mention it runs on almost every operating system natively without bogging your system down like a certain Media Player that comes with Windows? Who could ask for more??

Syncback V3 – I do a lot of work with Windows computers that are on the same network, many times I want to sync some directories between a few users, this will do just that, create a profile for each task you want, schedule it, run it, and forget it. Copies files even when their in use, options to make it overwrite files based on which is newer or older, or only sync files one way (more like a backup). I tend to forget I installed this on some machines as it doesn’t slow me down even if it’s running while I’m using the computer.

UltraVNC – Let me just say I’ve used VNC clients for years, this one is the most full featured, customizable, and yet the fastest. It will auto select connection settings, auto-scale the screen to fit the system your using. The server allows for silent installs and disabling user control, and will even hide the icon for you. Useful on systems you don’t want people knowing it’s running in the background.

MagicDisc – So you downloaded an ISO/BIN or pretty much any other type of compressed CD that you would normally be burning before you use. This little tool is a lifesaver sometimes, you can mount (acting as a virtual cdrom drive) almost all types of compressed CD formats to a virtual drive and use it without burning, even will install into the context menu (right click menu) so you don’t have to open the program first.

Notepad++ – Tired of notepad screwing up your perfectly formatted text/php/perl/ASCII files? Yea it gets old real fast, that when I wanted a quick replacement for notepad. Supports tabs, so you don’t end up with 10 or 20 open at a time. Other than that, just a basic text editor that won’t mess up our formatting, pretty straightforward, again, lightweight, and free. When downloading you’ll most likely want the .exe

Teracopy – Another replacement for a standard Windows tool. This will make coping files a breeze. How many times have you been coping files and Windows encounters a small error and the 20 minute file transfer cancels itself. Just once too many times it happened to me, I went in search and found this drop-in replacement for Windows copy. It will let you adjust the buffer size you use as well, has a few more bells and whistles, but I’m just glad to never see “could not copy xxxxx” and my whole transfer get canceled, as it will ask you what you want to do if it encounters an error, which doesn’t happen nearly as much.

I hope you enjoy these free tools as much as I do, this is what makes me able to stand using Windows for extended periods of time. None of these files should negatively effect Windows.

Let’s just start by explaining what each section does, below is some of my x10.sched:

timer smtwtfs 01/01-12/31 05:20 07:30 kitchen_on kitchen_off
macro kitchen_on 0 on a3-4
macro kitchen_off 0 off a3-4

The above will run a timer every day of the week, from 5:20 AM to 7:30AM. The kitchen_on command will be run at 5:20AM (when my fiance is getting up. The kitchen_off command will be run at 7:30AM when the sun has come up and she is normally gone to work. The macro section tells the system what you want the kitchen_on/kitchen_off to actually do. As you can see the first macro will turn devices A3 & A4 on, being my kitchen overhead light and the laundry room. The second macro will turn the same devices off. Pretty basic, but effective, the 0 in between the macro name and the command seems to be a default and is needed. Now onwards to triggers!

trigger a9 off a9off
macro a9off 0 off a1-2,6

The above will trigger the macro a9off when device A9 is turned off. It’s useful for tying whole living rooms together, and works vice versa for turning things on assuming you change the off commands to on. The macro a9off is set to control devices A1, A2, and A6.

Below is what I use for my outside lights, I want them to come on right about the time it gets dark outside and turn off about the time I usually go to bed. Who really likes walking up to your house, hands full and have to fumble for your key in the dark? As you can tell, it runs 7 days a week, the whole year. Now these look a little bit different than the previous, we have two macros running at the same time? The first macro on the first timer will turn the light on, the second macro will make sure it’s set to 100% power before we start to dim it, yes we’re going to dim these lights every day, I don’t see much point in using full bright lights, I want enough to see, but I can save a few bucks a months by toning down the power usage. So the second timer will run the macro that will dim the lights, as you can see the second time in the second time is 00:00 and the macro for that time is null, meaning it will do nothing. The third timer is the one that will turn off the light at 12:30am and again I use a null for the second macro since I don’t need it.

timer smtwtfs 01/01-12/31 17:45 17:45 outside_on outside_bright
timer smtwtfs 01/01-12/31 17:45 00:00 outside_dim null
timer smtwtfs 01/01-12/31 00:30 00:00 outside_off null
macro outside_on 0 on a5
macro outside_bright 0 bright a5 22
macro outside_dim 0 dim a5 9
macro outside_off 0 off a5

This timer below is kinda like a failsafe if I don’t turn out my lights, I run it every day except Friday and Saturday since I’m normally up later, and sometimes into the early and light AM.

timer smtwt.. 01/01-12/31 01:30 00:00 lightsout null
macro lightsout 0 off a2-4,6

This should give you a basic understanding on writing your own schedule, and using triggers to perhaps even ‘set the mood’ with the push of ONE button, or even a timer :-p

Now how do we make the CM11A use it? Well that’s easy, once you’ve typed it all up, save it to either /etc/heyu/x10.sched or /home/YOURHOME/.heyu/x10.sched. Then just open up a terminal on your box that has the CM11A and heyu and type heyu upload this should tell you it got uploaded successfully and your good to go.

On a side note, in the last entry there was a comment by Dan, visiting his page I saw some people showed interest in having an alarm clock that would slowly turn lights on. Well here it is, set a timer for the same time your alarm clock is set for, and you can even have it increase the brightness every few minutes. And there you have a homebrew way to make it seem like your alarm clock and lights are in cahoots together.

We’re still only scratching the surface of what linux can do when it’s tied to the rest of your world.
Click here to download a copy of my x10.sched

This requires:

1. Linux computer (I’m using Ubuntu 7.10, also works with CentOS5)
2. WS467 Modules – Their either 7 bucks a piece off eBay or 3 for 15 – Replacement wall switches
3. CM11a – About 10 bucks, runs the computer interface for the x10 control, stores macros. Uses a serial port, make sure you got one still!
4. Optional – Remotes -
   1. KR19A - This keychain remote control two devices, On/Off/Dim – Simple & small
   2. UR73A – This remote will do it all, don’t get up from your lazy boy just to turn off lights!
5. Very little experience with wiring.
6. 1 Flat-head & 1 Phillips screwdriver
7. A volt-meter is a GREAT investment, not necessary, but can eliminate questions and problems very quickly.

Let’s start with the CM11A. This little module will be the heartbeat to the whole system. You’ll want to plug in directly to the wall, try to avoid power strips and such. Plug it in to your linux box. Now we need the software to run it, I use heyu, very effective command line tool. You can download from http://heyu.tanj.com/download/heyu-2.1.2.tgz
You can use the below to setup the program.

cd ~
wget http://heyu.tanj.com/download/heyu-2.1.2.tgz
tar -zxvf heyu-2.1.2.tgz
./Configure
make
make install (make sure your root or using sudo)

It should ask you where you would like to store the config file, I tend to use /etc/heyu.
The default housecode is set to A, so assuming you use A, the only thing you may need to change is the serial port which is set to /dev/ttyS0 by default, but normally it’s good to go. Let’s install a module and see what happens.

So, your got your WS467 out of the box and you can’t wait to see this in action, let’s just make this clear early, if you shock yourself, that’s your problem. That being said, let’s find the circuit in your breaker that your going to be working on. Clearly if your breaker is labeled this should be easy, if not, leave the light on, and turn off the circuit, if the light goes out, your ready to go, otherwise keep going until you got the right one. Once you’ve found it, unscrew the two screws holding in the plate, pull the plate off and you should see two more screws, remove those as well, and pull the switch out enough to see how the wires are attached, generally it’s just wrapped around a screw and tightened. You may also have a copper wire run with other two, that would be a ground wire, the X10 module has a connection for it on the side. You’ll want to take the two wires off the original switch and twist one wire of the X10 module to each one, and I can’t stress enough it’s important to have a GOOD connection, make sure the wire nut is twisted on very tight. Make sure to set the housecode (A,B,C…) and the unit number (1-16) and write this down somewhere. Now before you push it all back in the wall you may want to turn on the circuit, test the light and test the automation. Back to the computer! Pull up a terminal and type:

heyu on A# (change # to your module number)
—-Watch light—-
heyu off A#
—-Watch light—-

Hopefully you just saw the lights go on and off. If it did not, some common problems:

1. Check the serial port
2. Check to make sure you have the right unit number
3. The signal will degrade over distance, large houses may require boosters or repeaters to get from end to end.

So ahead and replace those other two switches to wanted to automate, carry the keychain with your keys, turn on your lights when you walk up. In time I’ll go into scheduling lights on and off at certain times, tying some of your modules to your bluetooth proximity, very cheap motion sensors, and more than just a couple light switches. Maybe even setup a special IVR on your Asterisk server to control your home while your away from home!

Sipura 2000 (you can still find them some places, if you can’t the Linksys PAP2-NA should work just fine assuming you get an unlocked one)

You will want an old clunker PC that has a CDROM and a hard drive you can format. The only thing it should require in the end is something a little faster than 500mhz, one power cord, and one network cord.
You will also want a cheap headset or microphone to test with your computer.

Your going to want some sort of SSH client. I use SSH Secure Shell from www.ssh.com (direct link). Putty will work fine as well.

What provider will you use?
Broadvoice – Steep activation fee, lots of complaints in the past, but nothing but good, reliable service for me the past 6 months. $20 a month for unlimited World (including something like 30 countries). Instant setup!
Axvoice – Good solid service for the price, occasionally get all circuits busy when dialing international or local. Good unlimited plan for $18.99 for US/CAN
Telasip – Solid slightly more cutting edge service, $14.95 for 1500 outbound US/CAN unlimited incoming, and can pass your own callerID (useful later)

Of course you don’t have to use any of these, there are tons of other providers, but I run all these everyday and can recommend them all. This guide
will also use these three for sample setups.

Now that you got all that, download the ISO from www.trixbox.org (direct link). Burn it off, and pop it in your clunker, turn it on and hope that power
supply still fires up ;-p. You’ll be presented very few options to go wrong with, it should just breeze through, ask a couple basics like the password,
then it should spit out the CD and reboot. At this point you should login as root and run ‘netconfig’. You want to give the box a static IP since you
will have to forward some ports to it later. Give it an IP that is out of the range of your DHCP on your router and set the nameserver and gateway to
your router’s IP. Type ‘ifdown eth0 && ifup eth0′ this should restart the network and connect you to the internet. You should not need a
keyboard anymore for your trixbox install.

Fire up Firefox (or IE if you swing that way) and point it to http://yourtrixbox
This should give you:

Go ahead and fire up SSH and connect as root to your asterisk server. Now the first thing you want to do is change your passwords. We have a couple to change. Run the following as root:

passwd-maint
passwd-amp
passwd-meetme

When you change the FreePBX password you’ll have to update two files as well.

nano -w /etc/asterisk/manager.conf

Find ‘secret = amp11′ under the [admin] section. Change ‘amp11′ to your new desired password. CTRL+X followed by Y to save and exit.

Now, we need to edit the /etc/amportal.conf to use our new password.

nano -w /etc/amportal.conf

Find the line that says ‘AMPMGRPASS=amp11′ and change the ‘amp11′ to the new password you just set. CTRL+X followed by Y to save and exit.

amportal restart

Chaning mySQL passwords

In the same terminal:

mysqladmin -u asteriskuser -p password newpass

The default pass should be amp109, replace newpass with your new password

Update /etc/amportal.conf to use our new password.

nano -w /etc/amportal.conf

Find the line that says ‘AMPDBPASS=amp109′ and change amp109 to your new password, CTRL+X followed by Y to save and exit.

Still more, /etc/asterisk/cdr_mysql.conf

nano -w /etc/asterisk/cdr_mysql.conf

Find the line that says ‘password=amp109′ and change the ‘amp109′ to the new password you just set. CTRL+X followed by ‘Y’ to save and exit.

service mysqld restart
amportal restart

To change the default mysql root password, do the following:

mysqladmin -u root -p password newpass

The default should be passw0rd (notice the zero), change newpass to your password you just set.

Phew! Just for security eh?

Now you should be able to login by press ‘switch’ at the top right of the Trixbox page, enter ‘maint’ for the username and your maint password you just set. Once in you should hover over the Asterisk tab and click FreePBX, find the Module Admin and Check for updates online, upgrade all you can, click process, then come back around and install all the modules you want, I generally leave out the game ;-p. Click process and then click the Apply Changes near the top of the screen, the window will grey out and the server should re-write it’s config files and restart (not the whole computer).

Setting Up Extensions!

Click the Extensions link in FreePBX, your going to be adding a Generic SIP Device. You only need to fill out the Display Name (House), Extension 2000, and make sure you set a secret for the device (a simple password). You can turn on voicemail if you’d like. Do the same for extension 2010 but make the display name Desktop (or Laptop).
Now you should have two extensions, 2000 (House) and 2010 (Desktop)

Let’s move on to Ring Groups, there is order to the chaos I’m doing these in.
We’re going to use Ring Groups to give our cellphone pseudo extension numbers. It’s going to allow us to forward calls directly to your cellphone.
Click on the Ring Groups and follow the below picture to add your first.
Make sure you change the 123456789 to your cell phone

Then do the same thing again, but use your wife/girlfriend’s cell phone number and make the Ring Group Number 3001
You should have then both terminate on no answer since your cellphone has voicemail.
Apply the settings and let’s move on.

Why don’t we connect your softphone so we can do some system recordings.
Download Xlite (Windows) (Linux)
Install and bring up the Sip Account Settings.
You need:
Display Name: Desktop
Username: 2010
Password: YOURSECRET
Auth. User: 2010
Domain: 192.168.x.x <— YOUR TRIXBOX IP

If everything goes well it will register and say your username in the display.
Now we’re going to head over to the system recordings page, enter your extension number 2010, dial *77, recording a message for your default
greeting, I find it helps to write it out before. IE: “Hi, you’ve reach Bob & Sally at 1800BOBSALLY. Press 1 for House, Press 2 for Bob Cell, Press 3 for Sally cell….”
If dialing *77 doesn’t perform as advertised just dial *99 and you get the same options.
Save the recording with your own name.
Now that’s all done we can build the IVR! Oh! One more thing, click the DISA link, enter Default for the name, enter a pin and save the settings.
Now Apply the settings and we’ll move on.

Building the IVR

For a small system like this, this is pretty simple and straight forward. Going to the IVR link in FreePBX you can fill out a name for IVR
I use something like House. You can leave the default settings until you get to the bottom, you should have 3 blank areas with some options to the right of them. Enter 1 in the first blank box, and select your extension for the House in your options to the right. Enter 2 in the next box, and select Bob Cell from the Ring Group option. Enter 3 in the next box, you guessed it, Sally cell.
Save the settings at the bottom, then click Increase options once. In the new box, enter 4 and select the DISA/Default option (…but I didn’t record an option for number 4……)
Save and Apply the settings.

Let’s pull you back to the ol’ SSH client as root:
ping all these servers and find the one that has the lowest ping time (1ms would be good)
proxy.lax.broadvoice.com
proxy.dca.broadvoice.com
proxy.mia.broadvoice.com
proxy.atl.broadvoice.com
proxy.chi.broadvoice.com
proxy.bos.broadvoice.com
proxy.nyc.broadvoice.com

Then, nano -w /etc/hosts and make a line like the following substitute the ip for your fastest:
12.34.56.78 sip.broadvoice.com

CTRL + O to save CTRL + X to exit

On to the trunks!

Trunks are kinda like the equivelent to a “line” in a Ma Bell world. It’s really going to be our connection to our SIP provider. For now let’s
assume you went with BroadVoice.

Set Outbound CallerID: NAME HERE <NUMBERHERE>
Max Channels: 2 (two calls total on the line at once)

Dial Rules (this will catch all 10 digit dialing and some):
1+NXXXXXXXXX
NXXNXXNXXX
1NXXNXXNXXX

Give the trunk a name like broadvoice

Peer Details:
authname=1234567890
canreinvite=no ;if your behind NAT
context=from-trunk
dtmf=inband
dtmfmode=inband
fromdomain=sip.broadvoice.com
fromuser=1234567890
host=sip.broadvoice.com
insecure=very
secret=YOURSECRET
type=peer
user=phone
username=1234567890

Blank the incoming fields and move down to the registration. It will be something like the following:
1234567890:PASSWORD@sip.broadvoice.com/1234567890

Save the settings & Apply.

Let’s setup the Outbound Routes
There is a default blank that we can just edit. In the dial rules this is what I have:
911
011.
1+NXXNXXXXXX
1800NXXXXXX
1866NXXXXXX
1877NXXXXXX
1888NXXXXXX
1NXXNXXXXXX
NXXNXXXXXX
NXX+NXXXXXX

This catches pretty much everything and throws it to my trunks. If you don’t have 911 or International in your plan remove 911 and 011.
At the bottom you can select your Broadvoice trunk, save & apply.

Now just go back to your Inbound Routes and change the default destination to the IVR and calls should end up there.
Pick up your cellphone and run a test.
Don’t forget with the SSH client you can type ‘asterisk -r’ as root and connect to the server to watch calls in real time.