loon package

Submodules

loon.classes module

Classes used in loon package

class loon.classes.Host(hostfile='/home/docs/.config/loon/host.json')

Bases: object

Representation of remote host

add(name, username, host, port=22, dry_run=False)

Add a remote host

Parameters:

• name – hostname alias, a string
• username – hostname, a string
• host – host ip address, a string
• port – host ip port, an integer
• dry_run – if True, dry run the code

Returns:

None

cmd(commands, _logger=None, run_file=False, data_dir=None, remote_file=False, dir='/tmp', prog=None, dry_run=False)

Run command(s) in active remote host using channel session Therefore, open_channel in connect method must be True before using it.

Parameters:

• commands – commands/scripts run on active remote host
• _logger – the logging logger
• run_file – if True, run scripts instead of commands
• data_dir – a path representing data directory
• remote_file – if True, collect input from remote host instead of local machine
• dir – Remote directory for storing local scripts
• prog – a string representing the program to run the commands
• dry_run – if True, dry run the code

Returns:

A string containing result information

connect(privatekey_file='~/.ssh/id_rsa', passphrase='', open_channel=True)

Connect active host and open a session

Parameters:

• privatekey_file – a string representing the path to the private key file
• passphrase – a string representing the password
• open_channel – if True, open the SSH channel

Returns:

None

delete(name, username, host, port=22, dry_run=False)

Delete a remote host

Parameters:

• name – hostname alias, a string
• username – hostname, a string
• host – host ip address, a string
• port – host ip port, an integer
• dry_run – if True, dry run the code

Returns:

None

download(source, destination, _logger, use_rsync=False, dry_run=False)

Download files to local machine from active remote host.

Currently, it is dependent on scp command.

Parameters:

• source – list of files (directories) in remote host
• destination – destination directory in local machine
• _logger – the logging logger
• use_rsync – if True, use rsync instead of scp
• dry_run – if True, dry run the code

Returns:

None

get_result(print_info=True)

Get result from executed channel

Parameters:print_info – if True, print information Returns:a string containing output from executed commands

host_check(name, username, host, port=22)

Check if a host exists

Parameters:

• name – hostname alias, a string
• username – hostname, a string
• host – host ip address, a string
• port – host ip port, an integer

Returns:

a list representing the host

list()

List all remote hosts

load_hosts()

Load hosts from file

rename(old, new, dry_run=False)

Rename host name

Parameters:

• old – a string representing the old host name alias
• new – a string representing the new host name alias
• dry_run – if True, dry run the code

Returns:

None

save_hosts()

Save hosts to file

switch(name, username, host, port=22, dry_run=False)

Switch active host

Parameters:

• name – hostname alias, a string
• username – hostname, a string
• host – host ip address, a string
• port – host ip port, an integer
• dry_run – if True, dry run the code

Returns:

None

upload(source, destination, _logger, use_rsync=False, dry_run=False)

Upload files to active remote host.

Currently, it is dependent on scp command.

Parameters:

• source – list of files (directories) in local machine
• destination – destination directory in remote host
• _logger – the logging logger
• use_rsync – if True, use rsync instead of scp
• dry_run – if True, dry run the code

Returns:

None

class loon.classes.PBS

Bases: object

Representation of PBS task

check(host, job_id, dry_run=False)

Check PBS task status

Parameters:

• host – a host object
• job_id – a string the job id
• dry_run – if True, dry run the code

Returns:

Job status

deploy(host, source, destination, _logger, use_rsync=False, dry_run=False)

Deploy target directory on the active remote host

Upload the target destination and then submit all *.pbs files.

Parameters:

• host – a host object
• source – a string representing the directory (contains .pbs files) to upload
• destination – a string representing the path on remote host
• _logger – the logging logger
• use_rsync – if True, use rsync instead of scp
• dry_run – if True, dry run the code

Returns:

None

gen_pbs(template, samplefile, mapfile, outdir, _logger, pbs_mode=True, dry_run=False)

Generate a batch of (script) files (PBS tasks) based on template and mapping file

Parameters:

• template – a string representing the path to the template file
• samplefile – a string representing the path to the sample file
• mapfile – a string representing the path to the mapping file
• outdir – a string representing the path to output directory
• _logger – the logging logger
• pbs_mode – if True, use PBS mode
• dry_run – if True, dry run the code

Returns:

None

gen_pbs_example(outdir, _logger, dry_run=False)

Generate example files for pbsgen command to specified directory

Parameters:

• outdir – a string representing the output directory
• _logger – the logging logger
• dry_run – if True, dry run the code

Returns:

None

gen_template(input, output, dry_run=False)

Generate a PBS template

Parameters:

• input – a string representing the path to template file
• output – a string representing the path to output file
• dyr_run – if True, dry run the code

Returns:

None

sub(host, tasks, remote, workdir, _logger, dry_run=False)

Submit pbs tasks

Parameters:

• host – a host object
• tasks – a list of PBS files, glob pattern is supported
• remote – if True, means that PBS task files are located at the active remote host
• workdir – a directory representing the working directory
• _logger – the logging logger
• dry_run – if True, dry run the code

Returns:

A list of files

loon.skeleton module

The skeleton of loon package. Set command in [options.entry_points] section of setup.cfg file Install with python setup.py install

loon.skeleton.main(args)

Main entry point allowing external calls

Parameters:args ([str]) – command line parameter list

loon.skeleton.parse_args(args)

Parse command line parameters

Parameters:args ([str]) – command line parameters as list of strings Returns:command line parameters namespace Return type:argparse.Namespace

loon.skeleton.run()

Entry point for console_scripts

loon.skeleton.setup_logging(loglevel)

Setup basic logging

Parameters:loglevel (int) – minimum loglevel for emitting messages

loon.tool module

Tool functions

loon.tool.batch(input, cmds, sep=', ', header=False, thread=1, dry_run=False, _logger=None)

Batch process commands according to mappings from file

Parameters:

• input – stdin or a path to input file
• cmds – template command (with placeholder) to run
• sep – separator, default is ‘,’
• header – set True if input data contains a header line
• thread – number of threads to run in parallel
• dry_run – if True, dry run the code
• _logger – the logging logger

Returns:

None

loon.tool.prun(x)

loon.utils module

loon.utils.create_parentdir(path)

Create parent directory for a file/directory if not exists

Parameters:path ([str]) – a file path

loon.utils.decomment(csvfile)

loon.utils.get_filelist(dirName)

Create a list of file and sub directories names in the given directory.

Source code from: https://thispointer.com/python-how-to-get-list-of-files-in-directory-and-sub-directories/

loon.utils.pretty_table(title, content)

Print a pretty table according to the title and content

Parameters:

• title ([str]) – a list containing the title of table, e.g. [‘title1’, ‘title2’]
• content ([str]) – a nested list containing the content of table, e.g. [[‘a1’, ‘a2’], [‘b1’, ‘b2’]]

loon.utils.read_csv(file_path, sep=', ', rm_comment=True)

Read CSV file

Module contents