Know your tools

October 15th, 2011 No comments

Prerequisets

Linux/Unix host
Linux/Unix destination with SSH server

Ready, Set, Go!

Getting right down to it, here is a quick simple snippet I use quite often to transfer batches of files (often text, or other highly compress-able files) :

tar cz file.foo folder foo/* | ssh user@destination.host.com ‘cat > ~/file.tar.gz’

Today I found myself wanting to transfer an entire CD to a remote host, normally I’d just find a place to download it, but being a proprietary disc, that wasn’t an option.  This was too easy, but keep in mind if you don’t have a good upload speed, this can take awhile:

dd if=/dev/cdrom | ssh user@destination.host.com ‘cat > cd.iso’

Now of course you can mount it on the remote server:

mount /home/user/cd.iso -o loop /mount/path

Any questions?

Share
Categories: linux, Software, Uncategorized Tags:

The Mercurial Patch Queue

October 12th, 2011 No comments

Prerequisets

This article assumes you’re already familiar with Mercurial, if you’re not, there are plenty of places to find a quick and basic tutorial.  Chances are if you’ve ever used version control, you’ll be able to follow.  Some of the Mercurial commands used don’t show up until later versions of Mercurial (1.6), so if you’re running an old version (shoutout to all those Lenny users), there are ways to accomplish the same, but the syntax and ease-of-use is a little more difficult.

What is the patch queue?

Mercurial’s patch queue is probably seen differently depending on the environment it’s used in.  Personally, I like to think of it as “flexible changesets”, meaning until I’ve finalized my queue and pushed it upstream I can work on any revision in the queue and “refresh” the revision to include changes I make.

What does that mean?

If you’re familiar with version control, you know that generally speaking, changesets are immutable, if you wanted to add some extra comments to a file, you’d have to make them in a new revision, separate from the changes that make the comments relevant.  I like to keep my version control history clean and concise, and that is where the patch queue really shines.

The patch queue in the workplace

I work with many various Mercurial repositories on a daily basis and can have projects in-progress across multiple repositories.  Normally, I’m not going to finish a project in a single sitting, so the very first thing I do when I start working is create a patch, named for set of changes I’m planning to implement, but the name doesn’t really matter.

Example: hg qnew patch_queue_example

This initializes the patch queue and puts the “patch_queue_example” patch on top of the stack.  I am now free to make changes, and can then save them by “refreshing” the patch I’m in.

Example:
touch newfile
hg add newfile
echo “Test” >> newfile
hg qrefresh

Now, if you look at the head of the repository (hg heads), you’ll see a new changeset, with the patch name.

changeset:   0:6627f1eb79a5
tag:         patch_queue_example
tag:         qbase
tag:         qtip
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:14:59 2011 -0400
summary:     [mq]: patch_queue_example

This changeset includes my changes to “newfile”, which you can see by running “hg export tip”

Now, that I’ve made those changes, I want to work on changes to the same repository, but they don’t relate to my previous changeset, so I create a new patch.

hg qnew non_related_changes

Looking at “hg heads” again, I see a new revision:

changeset:   1:52cfaec4afa1
tag:         non_related_changes
tag:         qtip
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:19:38 2011 -0400
summary:     [mq]: non_related_changes

Now, I’ll go about making new changes:

echo “These changes are unrelated to my previous changes, so I’ve created a new patch” > unrelated_changes
hg add unrelated_changes
hg qrefresh (again, this is rolls the current changes into the current patch)
hg heads

You’ll notice the short revision id has stayed the same (1), but the node has changed

changeset:   1:52cfaec4afa1
tag:         non_related_changes
tag:         qtip
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:19:38 2011 -0400
summary:     [mq]: non_related_changes

You can see the changes by running “hg export tip”, you’ll see the “unrelated_changes” with the text we’ve echoed into it.

Great, get to the useful stuff

Now, let’s say I wanted to go back and edit the “patch_queue_example” changes, I decided I wanted to add some more files in that changeset.  Normally, you’d just add the files, and make a new commit, providing a commit message stating you forgot to do something in the previous commit, or something similar.
With the patch queue I can simple “pop-off” the current changeset and go back to editing “patch_queue_example”

hg qpop

popping non_related_changes
now at: patch_queue_example

So we’ll add a few files

echo “foo” > example1
echo “bar” > example2
echo “baz” > example3

Now, add them to Mercurial

hg add

adding example1
adding example2
adding example3

Looking at a “hg status”, you’ll see the files have been added:

A example1
A example2
A example3

Now, let’s save the changes and go back to the “unrelated_changes” patch

hg qrefresh
hg status

You’ll notice the files are no longer listed as added, because the changes have been “commited” ( != hg commit).

Let’s move back to the “unrelated_changes”, by “pushing” it back onto the top of the stack

hg qpush

applying non_related_changes
now at: non_related_changes

At this point, I hope you see where this is going, you can continue doing this until you’ve finished all the relevant work.
Now it’s time to convert them into “real changesets” so they can be pushed upstream, which I haven’t mentioned until now, you don’t want to try pushing patches upstream, namely because they should be considered unfinished until you’ve converted them, but also because each “hg qrefresh” you do, the node will be updated (Mercurial generates the node id based on changes/history, it will change if you change anything).

To convert these into real changes I want to do two things
1) Edit the commit messages
2) Remove them from the patch queue

Editing commit messages:
hg qpop (to move back to the first patch)
hg qrefresh -e (should drop you into a text editor, whatever you put here is the commit message)
hg qpush (to move to the second patch)
hg qrefresh -e (edit second patch commit message)

Removing from the patch queue:
Note: You can remove them one at a time using ‘hg qfinish $rev’, but we’re going to do it all at once.
hg qfinish -a

That’s it!  Now take a look at your log:

hg log

changeset:   1:1bb3052c69fc
tag:         tip
user:        Tim <example@example.com>
date:        Wed Oct 12 09:42:47 2011 -0400
summary:     This was the second commit in the Mercurial patch queue example

changeset:   0:5c616b8e4745
user:        Tim <example@example.com>
date:        Wed Oct 12 09:41:56 2011 -0400
summary:     This was the first patch in the Mercurial patch queue examples

These changes are ready to be pushed upstream.

This article of course does not cover extensive use and some of the neat things you can do while working in the patch queue, for the brave, here are some other commands I use frequently, maybe you can figure out where they’re good for.

Extra commands:
hg qnew -f new_patch_name
hg qpush –move patch_name

Share
Categories: development, linux, Software, version control Tags:

3 Favorite Geek Tips/Tricks

September 28th, 2009 No comments

Every geek should have at least a few tips or tricks that they don’t have to reference documentation or google for, here’s a few of my favorites that I use daily.

1) SSH Tunnels – Each and every time I’ve used this to cut half the time off a task I remember just how great SSH is.

When to use: Anytime you need internal access to a remote network. Or perhaps your company filters your internet usage and you need to check your gmail, this allows you to TUNNEL to your internet back at the house and ask it to do all the talking for you.

Example: I’m working from home, but I really need to access a ‘local’ resource on the remote network, it could be something as simple as connecting to the router on-site that doesn’t have remote access setup.

Howto Linux: If you’re a linux user, use the “-D” option with your standard ssh command. IE: ssh -l username -D 54321 sub.domain.com

Howto Windows: Get PuTTY. You’ll likely want to save this configuration so, open PuTTY, type in the hostname, give the session a name, on the left hand panel navigate to Connection->SSH->Tunnels. Under “Add New Forwarded Port” pick a port number, I like to use something high in the range, 54321 or you can pick your own. There are two rows of radio buttons, on the top choose “Dynamic” and leave the other as “Auto”, click “Add”. Now before connecting, on the left hand panel goto Sessions and click “Save”, now click “Open”

Utilizing the tunnel: What this essentially does is creates a secure SOCKS proxy for you to use. In Firefox, goto Preferences->Advanced->Network->Settings and set the SOCKS proxy address to “127.0.0.1″, that’s right, localhost (but don’t use ‘localhost’), then enter the port you chose earlier. Click “Ok” and you’re now surfing securely through SSH.

2) Converting movies to pocket size (linux only):
I have a simple Creative Zen 8GB Mp3 Player, it was cheap and works exactly as expected. It plays video of the same quality popular cell phones use, occasionally I want to watch a TV episode so I pieced together a simple little alias to make it quick and painless, you’ll need mencoder for this, if your on debian/ubuntu it’s as simple as ‘apt-get install mencoder’

Code (I put in ~/.bashrc): alias encode=’mencoder -ovc xvid -oac mp3lame -vf scale=320:240 -xvidencopts fixed_quant=4 -o ${args[0]} ${args[1]}’

Usage: encode outputFile.avi /path/to/file/to/convert.flv|avi|mpg

3) Using axel for super fast downloads
Situation: You’re remotely connected to a server you adminster, you’re crunched for time and need to download a file, maybe an ISO or perhaps a service pack, but you need it fast. You have no problem finding links to get it, but which do you use, which will be faster? The solution, grab 3 or 4 of those links and use axel.

Usage: axel -n X url1 url2 url3…
Replacing ‘X’ with the number of connections you want to use, passing as many URLs in as you’d like. In most cases, 3 or 4 seem to be enough to max out my 2MB/s.

Share

CLI Usefullness

August 30th, 2009 No comments

For the past year and a half I’ve been doing developement in a linux based enviroment, these tools and particularly these commands have become used frequently. If anyone has something useful, post it up!

Mercurial is a cross-platform, distributed revision control tool for software developers.

Easy way to view most recent changes:
Command: hg export tip | colordiff | less -R

Useful for: Quick code reviews, or just to peek at what you’ve recently done

Manipulating text files:

SED: (stream editor) is a Unix utility that (a) parses text files and (b) implements a programming language which can apply textual transformations to such files.

Command: sed ‘s/\t/”,”/g;s/^/”/;s/$/”/;s/\n//g’ > filename.csv
In action: mysql -uuser -ppass DBNAME -e “SELECT foo FROM bar WHERE baz = ‘world’” | sed .. > output.csv

Explanation:

  • “s/\t/”,”/g;” – Search for tabs, replace with commas
  • “s/$/”/;” – Replace last of line with quote
  • “s/\n//g” – Remove newlines

Useful for: This is specifically used for converting MySQL results to csv. I build lots of mailing lists for my employer.

AWK: is a programming language that is designed for processing text-based data, either in files or data streams, and was created at Bell Labs in the 1970s

Command: awk ‘BEGIN {OFS=”|” FS=”|”}; {print $1, $3, $5}’ file.csv
Explanation:

  • OFS – Output field separator
  • FS – Field separator
  • Printing fields 1, 3, and 5

Useful For: Pulling text out of delimited list and re-organizing the data, quickly and gui-free

Share

Avimark and linux, finally can dump windows server

July 9th, 2009 10 comments

There’s probably not that many people that actually care about this, but it’s taken me a long time to get to this point, so if your a tech guy at a vet’s office, enjoy! I’ve been running this successfully for almost a year now.

WARNING: If you choose to do this, that’s your choice, read this through before even thinking of diving in. In other words, I am not responsible for ANYTHING that happens as a result of what YOU choose to do. I also assume your familiar with linux and the distro of your choosing, really, I won’t be explaining some things as this is already going to be lengthy.

Note: This may not be the most secure way to do things, but I feel it’s stronger than windows, I have better backups (flyback for gnome, and a few scripts I’ve written), and no one has “Administrator” access, except me of course.

For the past 2 years or so I’ve been the ‘Tech Guy’ at two vet offices in my “free” time.  I’ve had one Windows 2000 server, which was actually the most reliable, compared to the Windows Small Business Server 2003.  I’ve experienced more problems than I care to recall, let’s suffice it to say, the workstations in the larger office were unreliable on network, even after a network rebuild (replace and tested all cable, made sure machines were clean, running the server on a gigabit network connection, all fun stuff).  Finally it got to the point where I was spending all my free time there, no good.  I am far more comfortable on any linux box than a windows machine, so I decided to take the plunge and make it work, installing linux servers that is. It may sound easy, but even since the switch I’ve had many unexpected surprises.  

Read more…

Share

Cha Cha Cha Changes

June 9th, 2008 No comments

Well, a lot has changed since the last time I posted anything here.  The biggest thing, getting married, new job, and even a new host.  Being married is great, the job is in the IT field, and the host is hostgator.com.  Funny story about switching host.

I signed up a few days ago, got the welcome email, and since I use zoneedit.com to manage the DNS (we’ll get to this later) I was in no hurry to get the DNS moved over.  I check my email last night and find an email from the hosting company saying I may be in violation of the TOS!  What?  I’m thinking I don’t even have a website there yet since I hadn’t switched the DNS, so I give them a call (mind you, they’re available 24/7 by phone via 1-800, very cool).  A guy picks up after waiting only a moment or so, again very cool, I explain the situation and ask how I could possibly be breaking the TOS before even have a website there.  The guy on the phone takes a moment to look up everything and understand what’s going on, they’ve flag it as a hacker web site.  “Really…. why?” I ask.  Of course at this point in my head I repeat the domain name, OH!  I think I understand now, so I suggest the guy on the phone visits the site to have a look for himself, which he directs to someone who can actually make the call.  We make small talk on the phone while we wait for the “reviewers”, the guy is very nice and rather knowledgeable, again very cool, he actually visits the site himself while we’re waiting and recognizes asterisk and x10!  After a few minutes the reviewers decided it was okay and instantly my account was reinstated, sweet..  This whole host switch was motivated by the terrible support and poor quality of totalchoicehosting.com, which I had been with for quite a few years, while other hosts improved over the years my host added nothing and kept the prices the same.  Just a small comparision:

TotalChoice: 2.4 gigs space
80 gb a month transfer
no ssh
5$ a month

Gatorhost: 600gb
6000 gb a month transfer
SSH!
8$ a month

So even if I would have bumped up to the 8.95$ plan with totalchoice the space went up to 3gb and transfer to 140gb a month, and still no ssh, still way less than I’m getting and already and
the support with hostgator has been great!

Now to get back to zoneedit.com.  This company has been around for quite some time now, I think I’ve been with them at least 8 or 9 years now, for free.  They offer a DNS service that rivals anything you’ve ever used.  The site may not look like much, but they let you manage up to 5 free “zones”, or domains.  So when you buy a new domain name, you use the nameservers they give you and they let you manage your dns settings, but heres the catch, it’s nearly INSTANT, anyone who ever has changed host knows that in the 24-72 hours after changing your website can be all out of sorts.  Not only do your changes take effect very quickly, you can add sub-domains that point to different ips, or even point something like foo.hackyourworld.com to google.com without the viewer even seeing the google.com.  If you find yourself wanting more control over your domain name, this is the ticket!

With all the changes I am planning on making this more regular, topics for the future will likely include bash, testing web pages, using ssh for demonstrations and opening your garage door by bluetooth proximity.  Hope you enjoy.

Share

My favorite 10 windows tools – another list!

December 14th, 2007 2 comments

I know I’ve been posting a lot today, there was just so much I had got half done and I’ve got the next few days off from work, but my fiance does not, so you’ll probably see even a couple more over the next few days, then you can expect it to slow down to 1-2 a week. So yea, it’s been done over and over, but I never seem to see many tools I use on this list, so just some common things I use on windows. I tend to look for the lightest but powerful tools, seeing how I know what happens to Windows when you stress it out, I keep my laptop running smooth and snappy using these tools. In no particular order:

* Looking at the stats from the traffic of this site we’re going to assume everyone already uses Firefox.

Read more…

Share
Categories: Freeware, Software, Windows Tags:

X10 Scheduling

December 13th, 2007 1 comment

In the last entry we just did some basics of the X10 system, now we’ll go over using them efficiently. The CM11A device allows you to upload a schedule to it that it will store in it’s own memory, so even if your computer is off, your X10 still works without missing a beat. All it takes is a small config file stored in /etc/heyu (or in ~/.heyu) called x10.sched . Heyu comes with a sample file, it’s very lengthy and can be confusing to some people.

Read more…

Share
Categories: Software, Ubuntu, X10 Tags:

$50 Home Automation

November 27th, 2007 10 comments

So who wouldn’t love to pull up to your house, get out of your car, walk in your house and all the lights and devices you love are already on? Or maybe your just want to save on your electric bill, either way with about 50 bucks you’ll be on your way to home automation paradise. This guide is not exactly intended from someone not at least kinda familiar with linux, and you should have at least used a screwdriver once or twice.

This requires:

  1. Linux computer (I’m using Ubuntu 7.10, also works with CentOS5)
  2. WS467 Modules – Their either 7 bucks a piece off eBay or 3 for 15 – Replacement wall switches
  3. CM11a – About 10 bucks, runs the computer interface for the x10 control, stores macros. Uses a serial port, make sure you got one still!
  4. Optional – Remotes -
    1. KR19A - This keychain remote control two devices, On/Off/Dim – Simple & small
    2. UR73A – This remote will do it all, don’t get up from your lazy boy just to turn off lights!
  5. Very little experience with wiring.
  6. 1 Flat-head & 1 Phillips screwdriver
  7. A volt-meter is a GREAT investment, not necessary, but can eliminate questions and problems very quickly.

Read more…

Share
Categories: Software, Ubuntu, X10 Tags:

Trixbox – Asterisk 1.4 & FreePBX 2.3.0 – w00t

September 24th, 2007 1 comment

We’re going to cover a lot of ground, this ‘guide’ assumes you at least know what linux is and hopefully at least used it once or twice.
The intention here is a complete walk thru to get a Trixbox system (2.2 or higher) up an running for a home setup. Some hardware/software you might want:

Sipura 2000 (you can still find them some places, if you can’t the Linksys PAP2-NA should work just fine assuming you get an unlocked one)

You will want an old clunker PC that has a CDROM and a hard drive you can format. The only thing it should require in the end is something a little faster than 500mhz, one power cord, and one network cord.
You will also want a cheap headset or microphone to test with your computer.

Your going to want some sort of SSH client. I use SSH Secure Shell from www.ssh.com (direct link). Putty will work fine as well.

Read more…

Share
Categories: Asterisk, Software, Trixbox Tags: