|
|
|
Other administrative tasks | Doc home | Madrigal home |
Some other tasks a Madrigal administrator might need to do:
Contact the OpenMadrigal administrator if some other administrative issue arises.
When users download ascii or netCDF4 versions of CEDAR Madrigal Hdf5 files as is (that is, just the parameters in the file without filtering), these alternative file formats by default are dynamically created. If this takes too long, the Madrigal admistrator can speed that up by creating cached files in the <experiment directory>/overview directory. The cached text file has '.txt.gz' appended to the basename of the CEDAR Madrigal Hdf5 file, and the netCDF4 file has '.nc' appended to the basename of the CEDAR Madrigal Hdf5 file. These cached files are created with the $MADROOT/bin/createCachedFiles.py script. Here's the usage:
createCachedFiles.py [--excludeText --excludeNetCDF4 --inst=--path= --includeNonDefault --overwrite --includeGeo --numCPU= -h --help] By default, both text and netCDF4 files created. Use --excludeText or --excludeNetCDF4 to only create one type. By default all instruments will be included. Use --inst= to only include some instruments. By default, all experiment directories will be included. Use --path to limit to a particular directory and all subdirectories. By default only default files will be cached. Use --includeNonDefault to include all files. Use --overwrite to overwrite all Hdf5 cached files. Default is to skip existing cached files. Set --includeGeo to also convert geophysical files. Default is to skip them. numCPU by default is the maximum of (1, numCPUs available - 2). Use --numCPU to override, but still will not be highter than default. -h or --help - print usage and exit
This new feature of Madrigal allows data files to be made either publicly available, or restricted to a limited set of IP address listed in a file.
To disable this feature and make all your data public, run the following from the $MADROOT directory:
$MADROOT/bin/setAccess $MADROOT/experiments public
To enable this feature, you must first create a file called "trustedIPs.txt" under MADROOT. This file should contain the IP addresses of hosts considered to be part of the private group, or partial IPs if whole sub-nets should be included. This file is not created during installation. Anyone whose browser's IP matches this list will be able to view files marked as private. For example, if trustedIPs.txt contains the line 132.197.*, any IP address that begins 132.197 will have private access.
Also, if the fileTab.txt file for that experiment can be overwritten by the web server, anyone whose browser's IP matches this list will be able to to change the access of any file between public and private from the madExperiments listing page.
To change a large number of experiments to be either public or private, run the following script from the $MADROOT directory:
$MADROOT/bin/setAccess dirPath [public || private]
where dirPath is the full path name of any directory in $MADROOT/experiments, and the second argument is either public or private. This script will set all experiments in dirPath and below to be public or private.
The access permission for any given file is set in the fileTab.txt file. See metadata documentation for details.
You can easily modify the Madrigal homepage of you site by modifying the template file $MADROOT/source/madpy/djangoMad/madweb/templates/madweb/index.html.
The following settings in the madroot/madrigal.cfg file can be modified at any time and the changes will take effect immediately:
If you want to add any documentation pages specific to your site to the Madrigal documentation pages, simply put them at a url where they can be indexed. Then edit the file MADROOT/source/madpy/djangoMad/madweb/templates/madweb/madContents.html to modify the line
<a href="#">
to replace # with the url where your site specific documentation is.
By default, users see only the Cedar rules-of-the-road on the data access page when they first enter the Madrigal site. If you want your users to see rules-of the-road specific to your Madrigal site when the reach the Madrigal experiment page, simply create a text file with your rules called local_rules_of_the_road.txt in the madroot directory.
Soemtimes its helpful to add static pages to your Madrigal site, such as a page describing your instrument, that you can then simply reference from each individual experiment page. To do this, simply copy the static file to MADROOT/source/madpy/djangoMad/madweb/static/siteSpecific, and use the url "/static/siteSpecific/[filename]" to refer to it.
If you want to copy data from one Madrigal site into yours (such as is done during the standard installation with the test data from Haystack) copy all the desired data from another Madrigal site into the appropriate directory in $MADROOT/experiments. Then follow these steps:
The script configureExperiments automatically edits the metadata files, changing the site id to your site id. This same procedure also works for moving experiments between experiments[0-9]* directories.
Sometimes the experiment start and end times in the metadata (the expTab.txt file) are manually edited, and do not match the start and end times of the data found in the files. The script updateExpTimes.py will examine the start and end times of all the files in an experiment, and set the experiment metadata start and end times to be the earliest and latest times found. Usage:
$MADROOT/bin/updateExpTimes.py [experiment_directory]
If optional argument experiment_directory given, then it will only update all experiments found in that directory. Default is to update the entire database. Only default or real time files are used to determine the times. Example:
$MADROOT/bin/updateExpTimes.py $MADROOT/experiments/1998
will update all experiment times for the year 1998.
With Madrigal 2.6, users can register interest in an existing experiment or in an instrument. When a data file is updated in that experiment using the standard admin scripts, by default all users registered for that experiment or instrument are sent an email. To ensure that is possible, you simply need to make sure the files $MADROOT/metadata/userdata/regExp.txt and $MADROOT/metadata/userdata/regExp.txt are writable by the web server. If you get a request from a user to unregister them from an experiment or an instrument, simply delete that line with their email from that file. With Madrigal 3.0, you can also use the utility $MADROOT/bin/registrationAdmin.py. Usage is:
registrationAdmin.py [--unregister | --register] [--inst=<kinst> | --exp=<expDir>] userEmail
Either unregister or register must be set. Either --inst=<instrument code> or --exp=<expDir> must be set. expDir is in form <experiments/2010/mlh/18jan10>
With Madrigal 2.6, administrators can expand the disc space available for experiments by mounting a directory named experiments[0-9]* in the Madroot directory. For example, you can add a directory called experiments2 in the Madroot directory and use it for a mount point for additional space for Madrigal experiments. After you create this additional mount, you will need to create a soft link in your web server document root with a soft link to that new directory. For example, say you do the following:
You can now make use of this new directory in two ways:
If you receive too many emails, you can use a third party tool to throttle the amount. The tool I have used successfully is django-email-throttler, which is described at https://github.com/robert-kisteleki/django-email-throttler. To install, use pip and then edit your settings_production.py file:
pip install django-email-throttler
Then add the following lines to the end of your file MADROOT/source/madpy/djangoMad/djangoMad/settings_production.py. The top lines can be adjusted to set exactly how many emails you get, but these numbers should work fine. When done, you will need to restart apache for the changes to take affect.
# stuff for email throttling and their defaults
EMAILTHROTTLER_TMPDIR = '/tmp'
EMAILTHROTTLER_PREFIX = 'emailthrottler-'
EMAILTHROTTLER_INTERVAL = 600
EMAILTHROTTLER_TRUNCATE_SUBJECT = None # 128 is also ok
EMAILTHROTTLER_SUBJECT_THRESHOLD = 3 # 0 for no-check
EMAILTHROTTLER_OVERALL_THRESHOLD = 6 # 0 for no-check
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'handlers': {
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler',
'email_backend': 'django_email_throttler.throttler.ThrottledEmailBackend',
'include_html': False,
}
},
'loggers': {
'django': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
|
|
|
Other administrative tasks | Doc home | Madrigal home |