Thursday, September 15, 2016

Setting up ctags - automatically for multiple independent projects

Howdy everyone.

For those of you using vim, you'll know that <C-]> will jump to a function definition.
The problem with this is that to jump between files searching for a definition, ctags files must be hanging around somewhere for vim to find them. There are plenty of guides out there to set it up, including the great guide at http://ctags.sourceforge.net/faq.html.

Unfortunately, this generates a huge tags file at the root of your projects directory saving references to all of your projects. If the projects are independent this is less than desirable.

This post describes how to set up separate tag trees for all of your projects independently.
Note that this assumes that each project is controlled by its own git repository. This method should be easily extendable to any project structure which has a predictable file/directory at its root and nowhere else in the project (if it's git controlled the .git directory fits this nicely).

We will accomplish this automation through 3 steps

  1. Build a script to generate hidden .tag files
  2. Add instructions to our .vimrc to update tags upon file save
  3. Automatically regenerate the full tag tree via cronjobs
1. Scripting the tag generation
-------------------------------------

The script can be found in the attached file updateTags.sh.

The important bits are the following:

TOP_DIR=$1
for rootdir in $(find "$TOP_DIR" -type d -name '.git'); do
  rootdir_nongit=$(echo $rootdir | sed -e 's|/[^/]*$||');
  for subdir in $(find $rootdir_nongit -type d \( -name '.*'                \
                        -o -name 'ext' -o -name 'doc' -o -name 'docs'       \
                        -o -name 'bin' \) -prune -o -type d -print); do
    cd "$subdir"
    ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl *;
  done
  cd $rootdir_nongit
  ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl    \
        --file-scope=no -R *;
done

The TOP_DIR variable merely is a convenience variable to represent the input, which should
be a directory. We then have the outermost loop, which cycles through all directories within $TOP_DIR which have a .git directory, and saves extracts the names of those variables to the rootdir_nongit variable.
The inner loop cycles through all subdirectories of $rootdir_nongit, and drops ctags files into them with the name .tags (because making them hidden prevents them from cluttering the file list for ls).
After all the subdirectories (except for hidden, ext, doc, docs and bin subdirectories) have ctags files created, a giant ctags file with references for the entire project is dropped in at the $rootdir_nongit level (this is the -R option).
You may want to modify the ctags options, see the man page for a complete description.

The attached file also contains a bunch of logging statements, and clears out any empty ctags files.

2. Telling VIM where to find the tag files
--------------------------------------------------
The following code should be added to your .vimrc file.

i) Set tags to the .tags file in the directory of the file you're editing
let &tags=expand('%:p:h')."/.tags"

ii) Find the .tags file in the root directory when entering a file
function! Setup_tags ()
  for rootDirs in finddir(".git", expand('%:p:h').";~/Projects",-1)
    if strlen(findfile('.tags',rootDirs[0:-5]))
      let &tags .= rootDirs[0:-5] . ".tags,"
    endif
  endfor
endfunction
autocmd VimEnter *.* :call Setup_tags()
This function finds the root directory (containing the .git directory, terminating the search for the root directory at the Projects directory), and adds it's .tags file to the list of tags locations to search. The final command outside of the function actually sets up the tags when vim is entered.

iii) Automatically update the tag file upon save
Since ctags is pretty quick, regenerating the tag file is quick and imperceptible, so I just like to regenerate the local .tags file upon file save.
To ensure that tag files are only updated (not generated if they are missing), add the following:
function! Update_tags ()
  if strlen(findfile('.tags', expand('%:p:h')))
    :silent !(cd %:p:h;ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl *)&
  endif
endfunction
autocmd BufWritePost * :call Update_tags()
the silent causes this to happen without prompting the user to acknowledge. The cd %:p:h ensures that the .tags file in the directory of the file you are editing gets regenerated - rather than the directory you entered vim from.

NOTE: If the files you work on are extra large, it may be better to just append the changes to the .tags file (which does not remove tags to functions that were deleted - hence the default of just regenerating the whole thing). To append, add the -a flag to the ctags call.

3. Automatically regenerate the .tags files
--------------------------------------------------
Automatically running scripts is trivial with the magic of cron. Since we have already created an updateTags.sh script, add the following line to your crontab (via crontab -e):
*/30 * * * * ~/updateTags.sh ~/Projects > ~/.local/logs/updateTags.log 2>&1
This line causes the tags to regenerate every half hour (change the 30 to 15 for quarter hour updates, etc). Furthermore, if you are using the linked file with all the logging, all the logged output will be dumped into your ~/.local/logs/updateTags.log file (make sure that the ~/.local/logs file exists before the cron job executes).

Setting up ctags - automatically for multiple independent projects

Howdy everyone.

For those of you using vim, you'll know that <C-]> will jump to a function definition.
The problem with this is that to jump between files searching for a definition, ctags files must be hanging around somewhere for vim to find them. There are plenty of guides out there to set it up, including the great guide at http://ctags.sourceforge.net/faq.html.

Unfortunately, this generates a huge tags file at the root of your projects directory saving references to all of your projects. If the projects are independent this is less than desirable.

This post describes how to set up separate tag trees for all of your projects independently.
Note that this assumes that each project is controlled by its own git repository. This method should be easily extendable to any project structure which has a predictable file/directory at its root and nowhere else in the project (if it's git controlled the .git directory fits this nicely).

We will accomplish this automation through 3 steps

  1. Build a script to generate hidden .tag files
  2. Add instructions to our .vimrc to update tags upon file save
  3. Automatically regenerate the full tag tree via cronjobs
1. Scripting the tag generation
-------------------------------------

The script can be found in the attached file updateTags.sh.

The important bits are the following:

TOP_DIR=$1
for rootdir in $(find "$TOP_DIR" -type d -name '.git'); do
  rootdir_nongit=$(echo $rootdir | sed -e 's|/[^/]*$||');
  for subdir in $(find $rootdir_nongit -type d \( -name '.*'                \
                        -o -name 'ext' -o -name 'doc' -o -name 'docs'       \
                        -o -name 'bin' \) -prune -o -type d -print); do
    cd "$subdir"
    ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl *;
  done
  cd $rootdir_nongit
  ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl    \
        --file-scope=no -R *;
done

The TOP_DIR variable merely is a convenience variable to represent the input, which should
be a directory. We then have the outermost loop, which cycles through all directories within $TOP_DIR which have a .git directory, and saves extracts the names of those variables to the rootdir_nongit variable.
The inner loop cycles through all subdirectories of $rootdir_nongit, and drops ctags files into them with the name .tags (because making them hidden prevents them from cluttering the file list for ls).
After all the subdirectories (except for hidden, ext, doc, docs and bin subdirectories) have ctags files created, a giant ctags file with references for the entire project is dropped in at the $rootdir_nongit level (this is the -R option).
You may want to modify the ctags options, see the man page for a complete description.

The attached file also contains a bunch of logging statements, and clears out any empty ctags files.

2. Telling VIM where to find the tag files
--------------------------------------------------
The following code should be added to your .vimrc file.

i) Set tags to the .tags file in the directory of the file you're editing
let &tags=expand('%:p:h')."/.tags"

ii) Find the .tags file in the root directory when entering a file
function! Setup_tags ()
  for rootDirs in finddir(".git", expand('%:p:h').";~/Projects",-1)
    if strlen(findfile('.tags',rootDirs[0:-5]))
      let &tags .= rootDirs[0:-5] . ".tags,"
    endif
  endfor
endfunction
autocmd VimEnter *.* :call Setup_tags()
This function finds the root directory (containing the .git directory, terminating the search for the root directory at the Projects directory), and adds it's .tags file to the list of tags locations to search. The final command outside of the function actually sets up the tags when vim is entered.

iii) Automatically update the tag file upon save
Since ctags is pretty quick, regenerating the tag file is quick and imperceptible, so I just like to regenerate the local .tags file upon file save, via the following line:
autocmd BufWritePost * :silent !(cd %:p:h;ctags -f .tags 
                                \--format=2 --excmd=mixed --extra=+q+f 
                                \--fields=nKsaSmtl *)&
the silent causes this to happen without prompting the user to acknowledge. The cd %:p:h ensures that the .tags file in the directory of the file you are editing gets regenerated - rather than the directory you entered vim from.

NOTE: If the files you work on are extra large, it may be better to just append the changes to the .tags file (which does not remove tags to functions that were deleted - hence the default of just regenerating the whole thing). To append, add the -a flag to the ctags call.

3. Automatically regenerate the .tags files
--------------------------------------------------
Automatically running scripts is trivial with the magic of cron. Since we have already created an updateTags.sh script, add the following line to your crontab (via crontab -e):
*/30 * * * * ~/updateTags.sh ~/Projects > ~/.local/logs/updateTags.log 2>&1
This line causes the tags to regenerate every half hour (change the 30 to 15 for quarter hour updates, etc). Furthermore, if you are using the linked file with all the logging, all the logged output will be dumped into your ~/.local/logs/updateTags.log file (make sure that the ~/.local/logs file exists before the cron job executes).

Saturday, May 14, 2016

Python source distribution with fortran dependencies

This post is intended make crystal clear how to distribute python packages which need to interface with a large fortran library. If you haven't already, you should familiarize yourself with the basics of using numpy.distutils with f2py [f2py-distutils documentation].

Are you back now - and fully understood the docs? Good.

Ideally, one can use f2py directives to eliminate the need for a signature file. What follows below will describe how to coax numpy.distutils into building the shared library WITHOUT having to create a signature file.


A Simple Example

Suppose you have the following Code structure (files and modules are 1 to 1 in this example):

src/
 |--pysrc/
 |    |--main.py
 |--forsrc/
 |    |--interface.f08
 |    |--codelogic.f08
 |--setup.py

codelogic.f08 may contain all sorts of derived types which f2py can't really handle. The interface.f08 file should contain subroutines and functions that contain ONLY arrays and default types (real, complex, integer, etc) within their message signatures (inputs and outputs). So we really want f2py to generate a shared library which accesses the interface module, but ignores the codelogic module.

The magic that makes this happen all occurs within the Extension object, inside an option called f2py_opts. Creating the following will do the trick:
    f2py_opts = [':','forsrc/interface.f08']
So within setup.py there will be an Extension object defined as:
    forlib = Extension(name='my_fort_lib',
                       sources=['forsrc/interface.f08',
                                'forsrc/codelogic.f08'],
                       f2py_opts=[':','forsrc/interface.f08'])
Toss this into your setup() function, and you should be able to
  import my_fort_lib
inside of main.py


A Deeper Look

Consider the following Extension:
  forlib = Extension(name='my_fort_lib',
                     sources=['forsrc/interface.f08',
                              'forsrc/codelogic.f08'],
                     extra_f90_compile_args=['-ggdb',-gbacktrace],
                     libraries=['m'],
                     f2py_opts=[':',
                                'forsrc/interface.f08',
                                'skip:',
                                'foo', 'bar'])
Here's a description of each keyword and what options were provided:

  •   name='my_fort_lib' -> The name of the shared library will be such that python can import my_fort_lib
  •   sources=[...] -> the fortran source files
  •   extra_f90_compile_args=[...] -> options passed to the compiler. -ggdb and -gbacktrace enable debugging and backtrace flags for the gcc compilers
  •   libraries -> libraries the linker needs to link against. m is the standard math library
  •   f2py_opts -> the options to pass to f2py
    • ':' -> the following items in the list become the only source files f2py uses when constructing the library. This terminates when another option is found in the list.
    • 'skip:' -> the following items in the list are skipped when constructing the library. These should be subroutine and function names.
    • 'foo', 'bar' -> subroutine names not exposed to the shared library available to python                  

Friday, May 1, 2015

An easy way to create a well formatted Journal/Diary

Ever wanted to start journaling your journey through life in a journal or diary?

I decided it would be a good idea, but had the following 2 issues:
1) writing it out by hand is a pain - keyboards are how I interact with the written world.
2) just jotting stuff down in a text or word file looks shabby.

So I fixed those 2 problems. I made a LaTeX document for which you can add an entry by simply typing make newEntry - then start recording your story from that day. Then whenever you feel that you want to read everything in a pretty format, type make and a well formatted PDF pops out that you can read through at your leisure.

Here's a fictional example with a couple of entries so you can see the format:
JournalExample.pdf

You'll note that the journal is divided into chapters. These can be set to whatever you like - an easy way to divide your life into successive stages for easy review in the future.

If you like the format, I have created a script which will generate everything needed to make your own Journal. Here's a link to the script:
journalScript.sh

Usage instructions (You need BASH. Linux [yes] and Mac [presumably]. Windows would take some work to set up):
1) Download journalScript.sh
2) move journalScript.sh to a new directory (not necessary, but its going to generate a lot of extra files which you probably don't want cluttering up anywhere else)
3) open a terminal, navigate to the directory, and run chmod +x journalScript.sh
It will prompt you for a title and your name.
Everything should now be set up - the README has instructions at this point.
4) type make newStage into the terminal - it will prompt you for a stage name (chapter title)
5) type make newEntry into the terminal - it will spit out a file location where you can record today's experiences - just open it with your favorite text editor and look for the '%start here' lines
6) repeat steps 4 and 5 (step 4 should be used rarely, whereas step 5 is ~daily) until you die

To view your journal:
1) navigate to the directory and type make into your terminal
2) open the generated PDF in your favorite PDF viewer

To get fancy with what you can include in your journal, check out the LaTeX Wikibook.

LaTeX Newbies: The important thing to note is that a new paragraph is only started if you leave a blank line. All lines between blank lines will be formatted into a single paragraph. So be feel to start all your sentences on consecutive lines, as it won't look weird in the final document.

Please note that this assumes that your environment has LaTeX and GNUMake installed on it.
Most Linux distros come with GNUMake, and LaTeX should be easy to find in your package repository. For Mac - you're on your own. But it shouldn't be too difficult.

Note: running this and just adding some empty stages and entries should provide a quick example of one way to organize a large multifile LaTeX document.

Monday, April 6, 2015

Open letter to congress on the renewal of the Patriot Act

Good morning congresspeople,

I am writing to emphasize the serious concerns arising due to the upcoming renewal of the Patriot act (due for renewal on June 1). Specifically, I would like to emphasize concerns with section 215. I find it incredibly broadly worded, and am worried about the consequences - especially concerning the privacy of US citizens.

In the upcoming process to renew it, please work with your congressional colleagues to eliminate this section, and replace it with specific enumerated powers for the NSA with regards to data collection. I strongly feel that those powers should be minimal. Ideally this would mean that they could not collect data without a warrant for data pertaining to a specific individual.

Even if the surveillance powers need to be much broader, I still urge you to work to enumerate such powers specifically rather than leave a broadly worded statement in the law.

Thank you for your time,
Peter Solfest

Wednesday, August 20, 2014

How to make a slideshow background for Ubuntu (scripted)

I have finally gotten sick of the backgrounds that come stock with Ubuntu, and wanted a slideshow.
Unfortunately, Ubuntu (I'm using 14.04 with a Unity interface) does not provide an easy way of doing this. So below is a script you can run to add a slideshow of images as a background option.

There are many graphical apps to do this (e.g., see this blog post 3-awesome-ubuntu-apps-for-wallpaper-slideshow/), but I a) like to know what's going on, b) am not a big fan of installing more apps than I need and c) discovered that this is a fairly easy problem to script away.

So without further ado, here is the script: MakeBackground.sh

How to use it
1) fill up a directory (henceforth known as <PicDir>) with images to loop through in the slideshow
2) download MakeBackground.sh
3) run '. MakeBackground.sh <PicDir> <Name>' in a terminal, where <Name> is the name you wish to give this slideshow (no spaces in <PicDir> or <Name>.

Details/Thoughts
============
-This background will break if you remove images from the <PicDir>, so leave it somewhere where you are unlikely to move pictures out of it (or leave a reminder for yourself)

-This script in and of itself works and sets up 15 minutes between transitions which last 5 seconds.

-This is kind of a scriptified version of the information found on https://help.ubuntu.com/community/SlideshowWallpapers

Friday, August 3, 2012

How To Create a Minecraft Desktop Launcher in Unity (Ubuntu 12.04)

I had recently decided to install and play minecraft on my desktop running Ubuntu 12.04, and after a couple of times got sick of having to open it up from the terminal all the time - so I decided to make a desktop shortcut. I figured I would post how I did this to save others some time. This should also allow for the easy creation of desktop shortcuts for other programs as well.

To make this work make sure that you know where the minecraft.jar file is and download an image for the desktop icon. (You can download the jar at http://www.minecraft.net/download. I used the icon at the following link http://fav.me/d36xhh8)

The first thing you need to do is create a .desktop file on your desktop. Copy/paste the following into your favorite text editor:

[Desktop Entry]
Name=Minecraft
GenericName=mcLaunch
Comment=Launches Minecraft
Type=Application
Exec=java -jar <Directory>/minecraft.jar -Xms 4096M
Icon=<Directory>/minecraftIcon.png
Terminal=false
Categories=Game;
Name[en_US]=Minecraft

for a full explanation of what all this does see Anatomy of a .desktop File.
Make sure to replace the bolded values with the appropriate directory (in place of directory) and memory allocation (in place of 4096M - 4 GB of RAM dedicated). Furthermore, if your image name is different make sure to replace it.

Save this file in your ~/Desktop/ directory with the filename Minecraft.desktop

The next thing you need to do is make sure that the file is executable. The easiest way to do this is right click on the desktop icon, select properties -> Permissions and fill in the box next to "Allow executing file as program". Alternatively, open up a terminal, cd into the Desktop directory, and type the following comand:
sudo chmod +x Minecraft.desktop

When you do this the icon you selected should be visible on your desktop, and double clicking it will launch minecraft.