Skip to content
Snippets Groups Projects
Commit 64e201f5 authored by Jan Kotanski's avatar Jan Kotanski
Browse files

New upstream version 0.2.3

parent 3295dd86
No related branches found
No related tags found
No related merge requests found
Showing
with 410 additions and 234 deletions
Settings : :mod:`settings`
###############################
source code settings
********************
.. automodule:: punx.settings
:members:
:synopsis: manage the settings file for this application
User interface : :mod:`main`
############################
Provides the user interface(s) to the punx program.
source code settings
********************
.. automodule:: punx.main
:members:
:synopsis: manage the application's user interface
NXDL Manager : :mod:`nxdl_manager`
##################################
source code documentation
*************************
.. automodule:: punx.nxdl_manager
:members:
:synopsis: Load and/or document the structure of a NeXus NXDL class specification
NXDL Rules: The XML Schema files : :mod:`nxdl_schema`
#####################################################
Read the NeXus XML Schema
----
source code documentation
*************************
.. automodule:: punx.nxdl_schema
:members:
:synopsis: Read the NeXus XML Schema
.. _nxdltree:
NXDL Definition File Tree Structure : :mod:`nxdltree`
#####################################################
Describe the tree structure of a NeXus Definition Language NXDL XML file.
Note: The *tree* subcommand replaces the now-legacy *structure* subcommand.
----
source code documentation
*************************
.. automodule:: punx.nxdltree
:members:
:synopsis: Describe the tree structure of a NXDL XML file
Manage the XML Schema files : :mod:`schema_manager`
###################################################
-tba-
--------
source code documentation
*************************
.. automodule:: punx.schema_manager
:members:
:synopsis: manages the XML Schema of this project
.. _source.validate:
Validation : :mod:`validate`
############################
.. index:: validation
.. index:: severity
The process of validation compares each item in an HDF5 data file
and compares it with the NeXus standards to check that the item is valid
within that standard. Each test is assigned a *finding* result, a
:class:`~punx.finding.Severity` object, with values and meanings
as shown in the table below.
======= ========= ==========================================================
value color meaning
======= ========= ==========================================================
OK green meets NeXus specification
NOTE palegreen does not meet NeXus specification, but acceptable
WARN yellow does not meet NeXus specification, not generally acceptable
ERROR red violates NeXus specification
TODO blue validation not implemented yet
UNUSED grey optional NeXus item not used in data file
COMMENT grey comment from the *punx* source code
======= ========= ==========================================================
Items marked with the WARN *severity* status are as noted in either the
NeXus manual [#]_, the NXDL language specification [#]_, or
the NeXus Definition Language (NXDL) files [#]_.
The *color* is a suggestion for use in a GUI.
Numerical values are associated with each finding value.
The sum of these values is averaged to produce a numerical
indication of the validation of the file against the NeXus standard.
An average of 100 indicates that the file meets the NeXus
specification for every validation test applied.
An average that is less than zero indicates that the
file contains content that is not valid with the NeXus standard.
NeXus HDF5 Data Files
---------------------
NeXus data files are HDF5 [#]_ and are validated against the suite of NXDL files
using tools provided by this package. The strategy is to compare the structure
of the HDF file with the structure of the NXDL file(s) as specified by the
``NX_class`` attributes of the various HDF groups in the data file.
NeXus NXDL Definition Language Files
------------------------------------
NXDL files are XML and are validated against the XML Schema file: ``nxdl.xsd``.
See the GitHub repository [#]_ for this file.
.. [#] NeXus manual:
http://download.nexusformat.org/doc/html/user_manual.html
.. [#] NXDL Language:
http://download.nexusformat.org/doc/html/nxdl.html
.. [#] NeXus Class Definitions (NXDL files):
http://download.nexusformat.org/doc/html/classes/index.html
.. [#] HDF5:
https://support.hdfgroup.org/HDF5/
.. [#] NeXus GitHub Definitions repository:
https://github.com/nexusformat/definitions
----
source code documentation
*************************
.. automodule:: punx.validate
:members:
:synopsis: validate NeXus NXDL and HDF5 data files
.. _structure:
.. _tree:
.. index:: tree
User interface: subcommand: **tree**
####################################
show tree structure of HDF5 or NXDL file
.. rubric:: command line help
.. code-block:: console
console> punx tree -h
usage: punx tree [-h] [-a] [-m MAX_ARRAY_ITEMS] infile
positional arguments:
infile HDF5 or NXDL file name
optional arguments:
-h, --help show this help message and exit
-a Do not print attributes of HDF5 file structure
-m MAX_ARRAY_ITEMS, --max_array_items MAX_ARRAY_ITEMS
maximum number of array items to be shown
Examples
++++++++
--tba--
.. _update:
.. index:: update
User interface: subcommand: **update**
######################################
**punx** keeps a local copy of the NeXus definition files.
The originals of these files are located on GitHub.
+.. caution:: The update process is being refactored, this may not work correctly now
To *update* the local cache of NeXus definitions, run:
.. code-block:: console
console> punx update
INFO: get repo info: https://api.github.com/repos/nexusformat/definitions/commits
INFO: git sha: 8eb46e229f900d1e77e37c4b6ee6e0405efe099c
INFO: git iso8601: 2016-06-17T18:05:28Z
INFO: not updating NeXus definitions files
This shows the current cache was up to date. Here's an example
when the source cache needed to be updated:
.. code-block:: console
console> punx update
INFO: get repo info: https://api.github.com/repos/nexusformat/definitions/commits
INFO: git sha: 8eb46e229f900d1e77e37c4b6ee6e0405efe099c
INFO: git iso8601: 2016-06-17T18:05:28Z
INFO: updating NeXus definitions files
INFO: download: https://github.com/nexusformat/definitions/archive/master.zip
INFO: extract ZIP to: C:/Users/Pete/Documents/eclipse/punx/src/punx/cache
.. rubric:: command line help
.. code-block:: console
console> punx update -h
punx update -h
usage: punx update [-h] [-f]
optional arguments:
-h, --help show this help message and exit
-f, --force force update (if GitHub available)
Examples
********
--tba--
Problems
********
.. _github_api_rate_limit_exceeded:
GitHub API rate limit exceeded
==============================
A common problem happens when updating the NXDL definitions from GitHub.
Here's what it looks like::
$ python ./src/punx/main.py update --force
('INFO:', 'get repo info: https://api.github.com/repos/nexusformat/definitions/commits')
Traceback (most recent call last):
File "./src/punx/main.py", line 416, in <module>
main()
File "./src/punx/main.py", line 412, in main
args.func(args)
File "./src/punx/main.py", line 170, in func_update
cache.update_NXDL_Cache(force_update=args.force)
File "/home/travis/build/prjemian/punx/src/punx/cache.py", line 257, in update_NXDL_Cache
info = __get_github_info__() # check with GitHub first
File "/home/travis/build/prjemian/punx/src/punx/cache.py", line 246, in __get_github_info__
punx.GITHUB_NXDL_REPOSITORY)
File "/home/travis/build/prjemian/punx/src/punx/cache.py", line 228, in githubMasterInfo
raise punx.CannotUpdateFromGithubNow(msg)
punx.CannotUpdateFromGithubNow: API rate limit exceeded for nn.nn.nn.nn.
(But here's the good news: Authenticated requests get a higher rate limit.
Check out the documentation for more details.)
GitHub imposes a limit on the number of unauthenticated downloads per hour [#]_.
You can check your rate limit status [#]_. Mostly, this means try again later.
.. [#] "The rate limit allows you to make up to 60 requests per hour,
associated with your IP address",
https://developer.github.com/v3/#rate-limiting
.. [#] Status of GitHub API Rate Limit: https://developer.github.com/v3/rate_limit/
A GitHub issue has been raised to resolve this for the **punx** project. [#]_
.. [#] *update: cannot download NXDL files from GitHub #64,*
https://github.com/prjemian/punx/issues/64
.. _validate:
.. index:: validate, validation
Validation
##########
.. toctree::
:hidden:
data_file_validation
nxdl_file_validation
*Validation* is the process of comparing an object with a standard.
An important aspect of validation is the report of each aspect tested and whether
or not it complies with the standard. This is a useful and necessary step when
composing NeXus HDF5 data files or software that will read NeXus data files and when
building NeXus Definition Language (NXDL) files.
In NeXus, three basic types of object can be validated:
* :ref:`HDF5 data files <data_file_validation>` must comply with the specifications set forth in the
applicable NeXus base classes, application definitions, and contributed definitions.
* :ref:`NeXus NXDL files <nxdl_file_validation>` must comply with the
XML Schema files `nxdl.xsd` and `nxdlTypes.xsd`.
* **XML Schema files** must comply with the rules defined by the WWW3 consortium.
TODO: citation needed.
User interface: subcommand: **validate**
****************************************
validate a NeXus file
.. rubric:: command line help
.. code-block:: console
:linenos:
usage: punx validate [-h] [--report REPORT] [-l [LOGFILE]] [-i INTEREST]
infile
positional arguments:
infile HDF5 or NXDL file name
optional arguments:
-h, --help show this help message and exit
--report REPORT select which validation findings to report, choices:
COMMENT,ERROR,NOTE,OK,TODO,UNUSED,WARN
-l [LOGFILE], --logfile [LOGFILE]
log output to file (default: no log file)
-i INTEREST, --interest INTEREST
logging interest level (1 - 50), default=1 (Level 1)
The **REPORT** findings are as presented in the table above for each validation step.
The logging **INTEREST** levels are for output from the program,
..
For now, refer to the source code documentation: :ref:`source.validate`.
Examples
========
--tba--
name: punx
dependencies:
- h5py
- lxml
- numpy
- python=2.7.*
- pyqt
- requests
- sip
- sphinx
- pip:
- versioneer
- pyRestTable
- PyGithub
- sphinx_rtd_theme
#!/bin/bash
# file: local_coverage.sh
coverage run tests
coverage html
coverage report
pypi.bat 0 → 100644
# refactor and use twine now
# see: https://pypi.org/project/twine/
# refactor the ~/.pypirc file
# see: https://docs.python.org/3/distutils/packageindex.html#the-pypirc-file
conda install twine
# remove build and dist directories (or replace wild cards with appropriate version)
python setup.py sdist bdist_wheel
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
twine upload dist/*
......@@ -2,20 +2,19 @@
description-file = README.rst
[bdist_wheel]
universal = 1
universal=1
[egg_info]
tag_build =
tag_date = 0
tag_date = false
[aliases]
release = egg_info -RDb ''
[versioneer]
vcs = git
VCS = git
style = pep440
versionfile_source = src/punx/_version.py
versionfile_build = punx/_version.py
tag_prefix =
parentdir_prefix = punx-
tag_prefix =
parentdir_prefix = punx-
......@@ -23,7 +23,8 @@ import punx
verbose=1
long_description = open('README.rst', 'r').read()
README = os.path.join(os.path.dirname(__file__), 'README.rst')
long_description = open(README, 'r').read()
setup (
......@@ -41,7 +42,7 @@ setup (
platforms = 'any',
install_requires = punx.__install_requires__,
package_dir = {'': 'src'},
packages = ['punx', ],
packages = ['punx', 'punx/validations', ],
#packages=find_packages(),
package_data = {
'punx': [
......
Metadata-Version: 1.1
Name: punx
Version: 0.1.9
Summary: Python Utilities for NeXus
Home-page: http://punx.readthedocs.io
Author: Pete R. Jemian
Author-email: prjemian@gmail.com
License: Creative Commons Attribution 4.0 International Public License (see LICENSE file)
Description: ####
punx
####
Python Utilities for NeXus HDF5 files: validation, structure, hierarchy
* Validation of NeXus NXDL files
* Validation of NeXus HDF5 data files
* Display of NeXus HDF5 data file structure
* Display of NeXus base class hierarchy (stretch goal, graphical output)
NOTE: project is under initial construction
:author: Pete R. Jemian
:email: prjemian@gmail.com
:copyright: 2017, Pete R. Jemian
:license: Creative Commons Attribution 4.0 International Public License (see *LICENSE.txt*)
:URL: http://punx.readthedocs.io
:git: https://github.com/prjemian/punx
:PyPI: https://pypi.python.org/pypi/punx/
:TODO list: https://github.com/prjemian/punx/issues
:build badges:
.. see http://shields.io/ for more badge ideas
.. image:: https://travis-ci.org/prjemian/punx.svg?branch=master
:target: https://travis-ci.org/prjemian/punx
.. image:: https://coveralls.io/repos/github/prjemian/punx/badge.svg?branch=master
:target: https://coveralls.io/github/prjemian/punx?branch=master
:release badges:
.. image:: https://img.shields.io/github/tag/prjemian/punx.svg
:target: https://github.com/prjemian/punx/tags
.. image:: https://img.shields.io/github/release/prjemian/punx.svg
:target: https://github.com/prjemian/punx/releases
.. image:: https://img.shields.io/pypi/v/punx.svg
:target: https://pypi.python.org/pypi/punx/
:social badges:
.. image:: http://depsy.org/api/package/pypi/punx/badge.svg
:target: http://depsy.org/package/python/punx
.. image:: https://badges.gitter.im/punx-nexus/Lobby.svg
:target: https://gitter.im/punx-nexus/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
Keywords: NeXus,HDF5
Platform: any
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: Freely Distributable
Classifier: License :: Public Domain
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Chemistry
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Scientific/Engineering :: Interface Engine/Protocol Translator
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Topic :: Scientific/Engineering :: Visualization
Classifier: Topic :: Software Development
Classifier: Topic :: Utilities
MANIFEST.in
README.rst
setup.cfg
setup.py
versioneer.py
src/punx/LICENSE.txt
src/punx/__init__.py
src/punx/_version.py
src/punx/analyze.py
src/punx/cache.py
src/punx/cache_manager.py
src/punx/dev_main.py
src/punx/finding.py
src/punx/github_handler.py
src/punx/h5structure.py
src/punx/logs.py
src/punx/main.py
src/punx/nxdl_manager.py
src/punx/nxdl_rules.py
src/punx/nxdl_schema.py
src/punx/nxdlstructure.py
src/punx/schema_manager.py
src/punx/settings.py
src/punx/singletons.py
src/punx/validate.py
src/punx.egg-info/PKG-INFO
src/punx.egg-info/SOURCES.txt
src/punx.egg-info/dependency_links.txt
src/punx.egg-info/entry_points.txt
src/punx.egg-info/requires.txt
src/punx.egg-info/top_level.txt
src/punx/cache/v3.2/__github_info__.json
src/punx/cache/v3.2/nxdl.xsd
src/punx/cache/v3.2/nxdlTypes.xsd
src/punx/cache/v3.2/applications/NXarchive.nxdl.xml
src/punx/cache/v3.2/applications/NXarpes.nxdl.xml
src/punx/cache/v3.2/applications/NXcanSAS.nxdl.xml
src/punx/cache/v3.2/applications/NXdirecttof.nxdl.xml
src/punx/cache/v3.2/applications/NXfluo.nxdl.xml
src/punx/cache/v3.2/applications/NXindirecttof.nxdl.xml
src/punx/cache/v3.2/applications/NXiqproc.nxdl.xml
src/punx/cache/v3.2/applications/NXlauetof.nxdl.xml
src/punx/cache/v3.2/applications/NXmonopd.nxdl.xml
src/punx/cache/v3.2/applications/NXmx.nxdl.xml
src/punx/cache/v3.2/applications/NXrefscan.nxdl.xml
src/punx/cache/v3.2/applications/NXreftof.nxdl.xml
src/punx/cache/v3.2/applications/NXsas.nxdl.xml
src/punx/cache/v3.2/applications/NXsastof.nxdl.xml
src/punx/cache/v3.2/applications/NXscan.nxdl.xml
src/punx/cache/v3.2/applications/NXspe.nxdl.xml
src/punx/cache/v3.2/applications/NXsqom.nxdl.xml
src/punx/cache/v3.2/applications/NXstxm.nxdl.xml
src/punx/cache/v3.2/applications/NXtas.nxdl.xml
src/punx/cache/v3.2/applications/NXtofnpd.nxdl.xml
src/punx/cache/v3.2/applications/NXtofraw.nxdl.xml
src/punx/cache/v3.2/applications/NXtofsingle.nxdl.xml
src/punx/cache/v3.2/applications/NXtomo.nxdl.xml
src/punx/cache/v3.2/applications/NXtomophase.nxdl.xml
src/punx/cache/v3.2/applications/NXtomoproc.nxdl.xml
src/punx/cache/v3.2/applications/NXxas.nxdl.xml
src/punx/cache/v3.2/applications/NXxasproc.nxdl.xml
src/punx/cache/v3.2/applications/NXxbase.nxdl.xml
src/punx/cache/v3.2/applications/NXxeuler.nxdl.xml
src/punx/cache/v3.2/applications/NXxkappa.nxdl.xml
src/punx/cache/v3.2/applications/NXxlaue.nxdl.xml
src/punx/cache/v3.2/applications/NXxlaueplate.nxdl.xml
src/punx/cache/v3.2/applications/NXxnb.nxdl.xml
src/punx/cache/v3.2/applications/NXxrot.nxdl.xml
src/punx/cache/v3.2/applications/nxdlformat.xsl
src/punx/cache/v3.2/base_classes/NXaperture.nxdl.xml
src/punx/cache/v3.2/base_classes/NXattenuator.nxdl.xml
src/punx/cache/v3.2/base_classes/NXbeam.nxdl.xml
src/punx/cache/v3.2/base_classes/NXbeam_stop.nxdl.xml
src/punx/cache/v3.2/base_classes/NXbending_magnet.nxdl.xml
src/punx/cache/v3.2/base_classes/NXcapillary.nxdl.xml
src/punx/cache/v3.2/base_classes/NXcharacterization.nxdl.xml
src/punx/cache/v3.2/base_classes/NXcite.nxdl.xml
src/punx/cache/v3.2/base_classes/NXcollection.nxdl.xml
src/punx/cache/v3.2/base_classes/NXcollimator.nxdl.xml
src/punx/cache/v3.2/base_classes/NXcrystal.nxdl.xml
src/punx/cache/v3.2/base_classes/NXdata.nxdl.xml
src/punx/cache/v3.2/base_classes/NXdetector.nxdl.xml
src/punx/cache/v3.2/base_classes/NXdetector_group.nxdl.xml
src/punx/cache/v3.2/base_classes/NXdetector_module.nxdl.xml
src/punx/cache/v3.2/base_classes/NXdisk_chopper.nxdl.xml
src/punx/cache/v3.2/base_classes/NXentry.nxdl.xml
src/punx/cache/v3.2/base_classes/NXenvironment.nxdl.xml
src/punx/cache/v3.2/base_classes/NXevent_data.nxdl.xml
src/punx/cache/v3.2/base_classes/NXfermi_chopper.nxdl.xml
src/punx/cache/v3.2/base_classes/NXfilter.nxdl.xml
src/punx/cache/v3.2/base_classes/NXflipper.nxdl.xml
src/punx/cache/v3.2/base_classes/NXfresnel_zone_plate.nxdl.xml
src/punx/cache/v3.2/base_classes/NXgeometry.nxdl.xml
src/punx/cache/v3.2/base_classes/NXgrating.nxdl.xml
src/punx/cache/v3.2/base_classes/NXguide.nxdl.xml
src/punx/cache/v3.2/base_classes/NXinsertion_device.nxdl.xml
src/punx/cache/v3.2/base_classes/NXinstrument.nxdl.xml
src/punx/cache/v3.2/base_classes/NXlog.nxdl.xml
src/punx/cache/v3.2/base_classes/NXmirror.nxdl.xml
src/punx/cache/v3.2/base_classes/NXmoderator.nxdl.xml
src/punx/cache/v3.2/base_classes/NXmonitor.nxdl.xml
src/punx/cache/v3.2/base_classes/NXmonochromator.nxdl.xml
src/punx/cache/v3.2/base_classes/NXnote.nxdl.xml
src/punx/cache/v3.2/base_classes/NXobject.nxdl.xml
src/punx/cache/v3.2/base_classes/NXorientation.nxdl.xml
src/punx/cache/v3.2/base_classes/NXparameters.nxdl.xml
src/punx/cache/v3.2/base_classes/NXpinhole.nxdl.xml
src/punx/cache/v3.2/base_classes/NXpolarizer.nxdl.xml
src/punx/cache/v3.2/base_classes/NXpositioner.nxdl.xml
src/punx/cache/v3.2/base_classes/NXprocess.nxdl.xml
src/punx/cache/v3.2/base_classes/NXroot.nxdl.xml
src/punx/cache/v3.2/base_classes/NXsample.nxdl.xml
src/punx/cache/v3.2/base_classes/NXsample_component.nxdl.xml
src/punx/cache/v3.2/base_classes/NXsensor.nxdl.xml
src/punx/cache/v3.2/base_classes/NXshape.nxdl.xml
src/punx/cache/v3.2/base_classes/NXslit.nxdl.xml
src/punx/cache/v3.2/base_classes/NXsource.nxdl.xml
src/punx/cache/v3.2/base_classes/NXsubentry.nxdl.xml
src/punx/cache/v3.2/base_classes/NXtransformations.nxdl.xml
src/punx/cache/v3.2/base_classes/NXtranslation.nxdl.xml
src/punx/cache/v3.2/base_classes/NXuser.nxdl.xml
src/punx/cache/v3.2/base_classes/NXvelocity_selector.nxdl.xml
src/punx/cache/v3.2/base_classes/NXxraylens.nxdl.xml
src/punx/cache/v3.2/base_classes/nxdlformat.xsl
src/punx/cache/v3.2/contributed_definitions/NXcontainer.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXelectrostatic_kicker.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXmagnetic_kicker.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXquadrupole_magnet.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXreflections.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXseparator.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXsnsevent.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXsnshisto.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXsolenoid_magnet.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXspecdata.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/NXspin_rotator.nxdl.xml
src/punx/cache/v3.2/contributed_definitions/nxdlformat.xsl
src/punx/data/writer_1_3.hdf5
src/punx/data/writer_2_1.hdf5
\ No newline at end of file
[console_scripts]
punx = punx.main:main
PyGithub
h5py
lxml
numpy
pyRestTable
requests
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment