Also make sure all your plugins are byte-compiled, this can shorten loading time significantly. Not a problem for elpa/package.el plugins, but I have quite a few plugins managed manually (I'm still not using el-get, shame on me) and the way I byte compile them all at once is to place a cursor on a directory with them in dired and issuing:
C-0 M-x byte-recompile-directory
The C-0 prefix arg causes it to recompile everything without asking for confirmation for every file.
You could automate this with a function hung off the after-load-functions hook. Here's something off the top of my head:
(defun foo-compile-if-newer (path)
"A function suitable for hanging off `after-load-functions',
which will byte-compile an Emacs Lisp source file for which there
is no pre-existing compiled file, or there is a compiled file
older than the source."
(and (save-match-data
(not (null (string-match "el$" path))))
(let ((elc-path (byte-compile-dest-file path)))
(if (or (not (file-exists-p elc-path))
(time-less-p (nth 5 (file-attributes elc-path))
(nth 5 (file-attributes path))))
(progn
(message "Auto-%scompiling %s..."
(if (file-exists-p elc-path) "re" "") path)
(byte-compile-file path))))))
(add-hook 'after-load-functions
'foo-compile-if-newer)
Adding an exclusion list, if you feel the need for one, is left as an exercise for the reader.