Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
How Ruby is beating Python in the battle for the Soul of System Administration (devopsanywhere.blogspot.com)
73 points by bryanwb on Oct 3, 2011 | hide | past | favorite | 44 comments


I'm not persuaded that the arrow of causality has been drawn correctly here.

I.e., it seems rather that Ruby has gained a high profile in system administration, not due to any inherent characteristics of the language or library, but because Puppet and subsequently Chef happened to be written by people who wanted to use Ruby.

Based on TFA, this was a matter of taste. I can't, for example, see why it should particularly matter for system administration whether "‘len’ was a function instead of a method)." It doesn't. But the fact that the guy who went on to write a reasonably important tool preferred to do so in Ruby on the basis of such personal prejudices made Ruby important just insofar as the tool was important, and probably contributed to Chef being written in Ruby as well.

Most of the reasoning in this article is no better than complaining that len() is a builtin. I fail to see how Perl-golf style conciseness is inherently more "productive" (particularly when it makes it harder to understand and maintain operationally important software). I fail to see how the crushing burden of spelling out 'import re' makes regex unacceptably distant in Python. I fail to see how Ruby is inherently stress-relieving or better for people who use vi, and if you don't think there is magic in Python that is probably because you have not gotten that deeply into the language. All this is pretty spurious, I think.

And if I wanted Perl, then Perl is the best possible Perl, already familiar to tons of sysadmins; and lots of good things are happening in Perl development.

What isn't spurious is if you happen to like Ruby, even if only for stupid reasons like Luke's; or if you really want to work with a tool like Chef that requires you to write Ruby. Those are perfectly good reasons for using Ruby.

But multiple languages will be used into the far future. In reality, the reason that Ruby and Python (and for that matter Perl) are so frequently put head-to-head is because they are so very similar in their abilities. That's okay. Write what you like.


I fail to see how Perl-golf style conciseness is inherently more "productive" (particularly when it makes it harder to understand and maintain operationally important software).

Let me ask a simple question here, how does one possibly remember all the vi commands or emacs commands for that matter? The answer is no one sits down with sheet of paper and memorizes these sort of things. A person sits down and understands how to use a particular tool, and then later on he just looks up to some form of a reference. Over time this becomes just muscle memory.

This where tools like Perl/Awk/Sed and other Unix text processing tools win over Python. You have to spend some learning how to use them initially after which you get very productive with them. For most people who don't have exposure to how much one can push the combination of pipes and Unix text processing utilities its a little difficult to explain it in words. You have to just use it to feel how power full they are.

Often you can save days of effort writing lengthy programs and testing them. All you need to be is familiar with the Unix text processing utilities and know how sew them using pipes and you can amazing lot of work without writing any code at all.

Perl has a special distinction that it not only helps you do all this things seamlessly but also makes a great scripting language for any task imaginable today. You get one platform for quick scripts plus application development of almost any kind.

They didn't call Perl the swiss army knife and duct tape of the internet just like that. I don't think any other programming language has come to adapting practical software realities as much as Perl has.

Every other programming language forces you think in one way or tries to force its paradigm you. On the other hand, Perl bends towards your paradigm. That to me is more than sufficient reason to use and go back to Perl again, because adaptability ensures survival on the longer run.

A painter is never happy when the brush dictates his art. A painter is happy when the brush paints the way he wishes to dictate his art.


Perl has a special distinction that it not only helps you do all this things seamlessly but also makes a great scripting language for any task imaginable today. You get one platform for quick scripts plus application development of almost any kind.

Until someone else has to maintain your code. I've noticed that most folks who default to Perl tend to write Perl in a manner largely inconsistent with the next guy's. "Bending toward your paradigm" is not an inherent good if anyone else ever has to deal with your paradigm after you leave.

Consistency at 90% your-arbitrary-metric-of-quality is generally better than wild inconsistency at 100% your-arbitrary-metric-of-quality, at least when multiple people are around.


I don't think this is true. Have you ever read the code of a bigger Perl distribution. For example Mojolicious.

https://github.com/kraih/mojo

While I also like other languages and for example have great fun using CoffeeScript for my current projects Perl is really good when it comes to describing what the code actually does. This is a benefit of "there is more than one way to do it". At the beginning t is hard to understand this, if you come from other languages. There are lots of people writing C-like Perl code. It looks very static and usually extremely ugly. People see this stuff, for example in the documentation of various bindings, etc. and think "ugh, that's ugly".

Perl (designed by a linguist) is much more dynamic, like a natural language. So it allows you to describe things in an hard or easier way. In fact Perl was my first real language and it took a while until I understood how others work. Meanwhile I know a number languages and have seen various ways in which you can design a language. I can see why many people shun Perl. However, the main reason possibly is that the don't understand what it's concepts are about. They are very different. One can possibly compare this with the situation when you switch to a functional language, like Haskell or when you haven't used OO before.

Like with all languages I really recommend to learn it. Even, if you don't use it Perl _will_ widen your horizon.

Two tips, if you do: Modern Perl, is probably the best book for learning and it's available as a free online book (wish I had something like that, when I started). There are many very outdated and even completely wrong tutorials. Oh and "use diagnostics;".

It will take a while until you see it's flexibility, but it is like Perl would be a language inherit (or assimilate) all paradigms and concepts or like always had them and they were just slumbering.

Well, of course there are other things like CPAN that make Perl great. Yeah, other languages have similar facilities, but they don't come close. It is really great and they all have lots of tests and there are a lot of "CPAN Testers" that report problem. And when you want portability you'll love it. It runs on _very_ exotic platforms.


Thank you for the links and the advice, but it is misplaced. I know and use Perl. I also dread having to do so. These two statements are related.

"There's more than one way to do it" is, for me, a negative, at least in the Perl implementation of it; while there are cases where multiple ways to do the same thing is great (for example, C#'s foreach versus the IEnumerable extension method that takes a delegate/lambda), I don't think that's the case here. The pathological decisions toward terseness are, in my experience, the root cause of one guy's Perl being unreadable to the next; this may be a cynical statement, but (said with tongue firmly in cheek) when you give a programming language capable of expressing significant logic with one character, the guy who uses vim will do so, and damn the next guy who has to pick it up and work with it later.

Say what you will about something like Java--and oh man, have I said all of it--but the language does tend toward readability (at the cost of boilerplate); it's relatively hard to write correct code that will elicit WTFs from the next guy who has to maintain it. In my experience, WTFs are somewhat par for the course with inherited Perl.

(I find Python to generally encourage the Right Thing as well, though it's easier to write monstrosities in that, too, than I would really like. C#, for me, hits the sweet spot, but doesn't exactly work so well as a scripting language.)

None of this should be construed to say that you cannot write clean Perl. Surely you can, and I've certainly encountered some of it. But I think "there's more than one way to do it" is more of a crutch for "write byte vomit, it'll just work" than you'd like to admit.


Just so you know, there is somewhat of a movement afoot in Perl these days. Many of us long term Perl programmers know well the dangers of TIMTOWTDI. So we have created TIMTOWTDI-BSCINABTE (pronounced TimToady Bicarbonate) and stands for "But Sometimes Consistency Is Not A Bad Thing Either". This has come specifically out of the Moose community and basically pushes to try and spread consistent best practices throughout the community.

I think the issue you encounter is a combination of isolation (the more involved you are in the community, the more you are exposed to best-practices and idiomatic Perl) and the quality of some of the older Perl documentation and books. In some of these older documents there was an emphasis put on "cleverness through TIMTOWTDI" which really had a damaging effect on some people (read: the people whose code you have inherited).

Perl is an extremely flexible language, which in the right hands can be very powerful. However, as Uncle Ben said, with great power comes great responsibility. I personally would rather my language require a little more self discipline from me 90% of the time, so that in the remaining 10% of the time I can hack some really crazy shit to get the job done.


I've noticed that most folks who default to Perl tend to write Perl in a manner largely inconsistent with the next guy's.

I don't know how 'Perl crowd' is a distinct set of people. Speaking of programmers in general. If a group/person have/has bad habits/practices. That's going to spill over to any language they/he is ever going to use.

No single language is an exception to this. By the way yesterday there was thread going on about how some one lost all his changes because he didn't commit his code regularly and blamed the problem squarely on git.

Now here is the problem, how would have any other VCS saved from making this mistake. Languages do very little to influence a person habits.

Besides there isn't a single language on earth today in which you can't write bad code.


I fail to see how Perl-golf style conciseness is inherently more "productive" (particularly when it makes it harder to understand and maintain operationally important software).

I'm not sure the author made a great case, but it does matter. Unix admins often think in terms of lines, because lines are the default unit of action. No matter how much planning and cfengining you do, there are still times when an admin must take direct action on a system. When that happens, they use the interactive command lines. These lines often aren't repeated frequently and have no need to be stored in any file other than .bash_history, and therefore do not need to be read or maintained. In those situations, the conceptual and physical overhead incurred by something like 'import re' is most assuredly significant.


I can get insanely concise one-liners out of C and shell, but that doesn't make those inherently more-productive languages.

I forget what bash does, but when I write a block of shell in zsh, it is saved as a 'line.' This is a tool issue, not an issue of language expressiveness.

If you can't write concise Python, maybe that's just because you aren't that comfortable with Python... that's a perfectly good reason not to use it but it isn't some sort of dramatic limiting case requiring all system administrators to use Ruby instead.


I don't use Ruby at all, personally. I use Python and Bash almost exclusively. I use Puppet but am not a huge fan, and I certainly have no plans to switch to ruby. I mostly agree with your original point.

However, the CLI thing is a legitimate advantage in favor of Ruby and Perl. Python is just really annoying to use for system administration one-liners. A number of fundamental design choices that have a minimal impact on even the smallest .py files make 'python -c' cumbersome.

Specifically:

A number of common tasks available as syntax in Perl and Ruby are in libraries in python. In particular, you often need to import sys, os, re, and subprocess. Not a real issue writing scripts, but adds a lot of overhead to a single line.

You can't pass a DEDENT token to '-c', at least I haven't figured it out, meaning you can't use more than one loop or control structure. You can work around this to some extent using list comprehensions.

Python's string literal syntax is more limited. I have never had an issue with this writing scripts, however from the command line it can be annoyingly tricky to keep track of which quote characters are needed. Ruby and Perl both have non-conflicting options for notating string literals. In shell scripts, strings are the default literal and you only need to worry about keywords and special characters.

Another advantage of shell are list literals. (I don't have enough practice to know how Ruby and Perl fare in this regard).

Bash also has some extremely convenient list expansion syntax.

    for fqdn in {www,news}.ycombinator.com ; do echo $fqdn ; done

    python -c "import sys; [sys.stdout.write('%s\n' % fqdn) for fqdn in ['%s.ycombinator.com' % sd for sd in ['www', 'news']]]"
Obviously this particular example could technically be shorter, since I could use for-loop syntax (and in python 3+ print is a function). But I would have to arrange it like this if I needed another nested loop. If you wanted to perform some additional action on the names you might have to import yet another library.

Incidentally, C can be a useful admin tool if you know how to use it, but usually the compile-execute cycle makes it more cumbersome than it's worth for one-offs. A C compiler might not even be available on your production system.


see my summary at the end. The real cause of ruby's rise is that puppet was written in it and that you use ruby-based dsl to configure puppet. This factor is more significant than language differences.


you also seem to have missed the multiple times I state my personal preference for Python but am shifting to ruby for practical reasons.


This is a rather misguided article. Unless everything has completely changed, Ruby is used for almost nothing in mainstream Linux. Bash and Python dominate "stock" system management code, and Bash and Perl seem to dominate for top-level add-on scripts written by a given sysadmin.

I've never even heard of any of those Ruby projects besides puppet, and I've only ever once installed Ruby- to support some obscure v0.03a library a developer wanted to try.

(Was a part-time sysadmin for 2 years)


The unstated premise of this article is that all sysadmins will very soon be using puppet or chef, thus programming in some subset or DSL that looks a lot like ruby (puppet).

However, that's a separate article that I need to get around to writing.


Neither Python nor Ruby are best suited for one-liners that the author is emphasising. One liners are bash-fu or commandline-fu. Anything above 100 lines (just a figure from the article) is what Python or Ruby should be used for.

Last 10 lines in a file? 'tail -10 filename.txt'

Right tools for the right job.

As to Python vs Ruby? I believe the argument isn't about the language (although I think Python is more 'sane' :) ), but the community that surrounds it. And I like the Python community.


     Last 10 lines in a file? 'tail -10 filename.txt'
A lot uglier, but you can do a one liner ...

     ruby -e 'lines=[]; while gets(); lines << $_; \
       lines = lines[-10,10] unless $. < 10; end; \
       puts lines' < filename.txt
The problem with using "the right tool for the right job" is that there are many tools and many jobs, and a general purpose language helps you get things done in case you're stuck. In general I use Ruby for doing string manipulation before piping to other commands.

Here's an (ugly) one liner that shows the biggest tables in a MySQL database:

     mysql -u root godzilla -s -e 'show tables' \
     | ruby -ne '$_.strip!; puts %Q{SELECT count(*) \
       as cnt, "#$_" as tbl FROM #$_; }' \
     | mysql -u root godzilla -s \
     | sort -nr \
     | ruby -ne 'parts = $_.split /\s+/; \
       puts "%40s : %s" % parts.reverse' \
     | head -10
You can probably come up with something more clever, but I barely gave that one any thought.


> A lot uglier, but you can do a one liner ...

Yeah, but that one-liner reads the whole file. 'tail' can be more clever (and is). 'strace' output:

    open("a", O_RDONLY)                     = 3
    fstat(3, {st_mode=S_IFREG|0644, st_size=545975512, ...}) = 0
    lseek(3, 0, SEEK_CUR)                   = 0
    lseek(3, 0, SEEK_END)                   = 545975512
    lseek(3, 545972224, SEEK_SET)           = 545972224
    read(3, "tu 11.04 \\n \\l\n\nUbuntu 11.04 \\n "...,     3288) = 3288
'tail' is seeking to the end, then reading backwards.

The point being, these 'little tools' can contain important optimisations - they aren't necessarily equivalent to a naive reimplementation.


Reminds me on that article on GNU grep doing some speedy magic: http://lists.freebsd.org/pipermail/freebsd-current/2010-Augu...


tail can be done in ruby can be quite succinctly with the $< (ARGF) variable:

  ruby -e'$><<$<.map.last(10)'
For the rest of the ruby one-liners in the page the author references, most can be done more easily with standard command line tools (although most people aren't well-versed in sed, so 'ruby -pe puts' might be better than 'sed G').


There's one big problem with code like that. If you're doing devops-y stuff, then you really, really don't want to debug something like that when your pager goes off at 3am. The cleaner the solution the better, because after you wake up `$><<$<` is just a blurred thing with no meaning...


Cool. That hurts my eyes.

    ruby -e "print ARGF.readlines.last(10)"


on, say, a 5GB log file?..

If I read this correctly (http://www.ruby-doc.org/core/classes/IO.html#M000914) it'll read the whole file into an array, then spit out last 10 entries?


Yes it does - good point, but the sigil-tastic snippet in the parent reads the whole file into an array, too - it's just harder to tell.

Plus the parent snippet is not Ruby 1.9 compatible.

Anyways, my version was just to illustrate that sysadmin scripting in Ruby does not have illegible.


Not that you can't do this in python:

python -c "print '\n'.join(open('LICENSE.txt','r').read().split('\n')[:-10])"

But this is besides the point.


Your split and join seem redundnant, and open defaults to read-only.

  print open('whatever.file').readlines()[:-10]
But still, don't do that on any log too big to read into memory.


tac filename.txt | sed 10q | tac


As a sysadmin who uses Puppet for a few hundred servers, I strongly disagree with the assertion that "After spending 25% of your time working with Puppet, you will be much more likely to reach for ruby for your next scripting task." I'm not even sure how you reach this conclusion since Puppet uses a DSL and not Ruby. I've been using Puppet for a while now and I don't know and have never used Ruby, and I'm not planning to. Much of what you have written seems like you're really stretching to validate your decision to start using Ruby instead of Python, which I'm not even sure why it matters. Use what you/your organization prefers, otherwise it doesn't really matter.


I somewhat agree even as a Ruby fan of 14 years.

We have puppet now at work, and going through the recipes, its rather obvious at times that people can use Ruby, without really learning the language.

I managed to shorten a number of the recipes and configs from 60 lines of... well WTF basically, to 10-20ish lines of much more readable ruby.

I will however agree with the assertion that if the management tools that are most popular are written in Ruby, one would be more likely to do more within them.


This article should have the following title: "How Ruby is Beating Python in the Battle for my Soul as System Administration". The summary shows that the person has not been working in a team to manage many components:

> Ruby's greatest strength is its amazing flexibility. There is a lot of "magic" in ruby and sometimes it is dark magic. Python intentionally has minimal magic. It's greatest strengths are the best practices it enforces across its community. These practices make Python very readable across different projects; they ensure high quality documentation; they make the standard library kick ass. But the fact is that we sysadmins need flexibility more than we need raw power or consistency.

As a sysadmin you insanely need consistency all over the place or you cannot scale. This is in fact why Fabric, Puppet and Chef were created in the first place, to have consistent and automatic ways to deploy on and maintain your systems.


> As a sysadmin you insanely need consistency all over the place or you cannot scale.

This is exactly what ran through my had reading the bit you quoted above: consistency is king! Try scaling magic. Maybe some people are able to do this. I, personally, am a mere mortal, and that means consistent, clear code is a very powerful tool to me. More so than this "flexibility" the article's author is citing. In fact I would argue that consistency and clarity lend flexibility if you understand what you're doing.


I use puppet in a mostly Python shop. I like that it uses a DSL and not raw Ruby/Pyhton/etc. However, I diagree with TFA about Ruby becoming the dominant language for a very simple reason: all UNIX-like systems have sh and in recent ones it supports all the same things. Most distros which I would consider installing onto servers, managed workstations, etc. have a Python interpreter and a standard library which includes everything from process management to a web server. These two tools are ubiqutous. However, not many distros ship with Ruby installed. Writing a 200 line Ruby script means that it is noe your job to ensure you have Ruby on all your machines, not the dist maintainers'.


"Puppet and Chef are seeing rapid adoption" != "ruby is inexorably becoming the dominant scripting language for linux system administration."


The short/one-liner examples are more likely to be done from command-line shell, where they would be much easier:

  head -10 /path/to/file

  dmidecode | grep -iq vmware && echo "is vmware" || echo "is not vmware"
The list of projects written in each language has more to do with what else you use than the languages, e.g., if you are working with Ruby web apps you may use Rake and Capistrano. SCons, Mercurial, Bazaar, and YUM are all written in Python.


This essentially reduces to "it's easier to write a fluent, readable, flexible and powerful DSL in Ruby than it is in Python.

Sysadmin tools benefit greatly from using such a DSL as their interface vs custom parsers or configuration files.

Hence, Ruby is a good choice for these sorts of things.


There's one thing left to do: Write a better tool than chef or puppet and do it in Python. That or you know - use the correct tool for the job and be done with it.


The Python example is completely broken. It'd be something like:

    import os
    if 'vmware' in os.popen('dmidecode').upper():
        print 'this is a vmware vm'
    else:
        print 'this is not a vmware vm'
I like regexps as much as the next ex-Perl hacker, but sometimes a little string manipulation is a lot better.


The condition will always evaluate to False, a lower-case string can't be in a string that has been .uppper()ed. popen(...).lower() will work.


fixed, tks


Again, another article that has a lot of talk about CM systems (CFEngine, puppet, chef, etc) and no love for bcfg2.. which happens to be Python based.


After seeing backticks, %x and $? being used, I have to say Ruby, when employed like this, competes more with Perl than with Python.


Ruby and Python break backward compatibility too often even with minor language version upgrade. It's harder to maintain working language version across servers than do real sysadmin tasks.

Perl rocks and robust to that.

See Perl equvalent to Ruby's Capistrano/Puppet/Chef.. or Python's Fabric/func..

Rex - http://rexify.org/


Only a moron would refuse to use a good tool if it's written in a language that is not said moron's personal idol.

Except of course if the language is Java.


Apparently there is an unstated premise -- sysadmins using Powershell lack souls.


tl;dr; ruby is the new perl




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: