Requirements for SDSS Data Management Systems June 5, 1995
[All requirements are subject to budgetary, manpower, and technical
The data management systems shall maintain all processed data from the
SDSS and provide access by SDSS scientists and operators in order to maximize
the ease of the following:
- Operate the SDSS survey so as to maximize the efficiency of operations.
- Perform Quality Analysis operations on the data so as to ensure its
integrity. The operations will verify calibrations, target selection
criteria and classifications, and completeness & accuracy.
- Provide SDSS scientists with access to the data and tools to permit
selection of spectroscopic targets for certain categories (serendipity;
- Provide SDSS scientists with access to the data so as to enable
Requirements for the Science Archive
The science archive shall consist of:
- A science database that shall:
- a. Retain calibrated object catalogs (photometric CCD output)
- b. Retain parameters from spectroscopic pipeline
- c. Enhanced goal: provide ability to recalibrate
- d. Retain references to atlas images and extracted spectra
- e. Provide ability to carry out manual target selection
for certain target categories
- f. Provide ability to carry out offline QA activities
- g. Provide ability for SDSS scientists to extract subsets
of retained data.
- h. Enhanced goal: Provide ability to retain
scientist-derived data sets.
- i. Enhanced goal: Provide smooth transition to
public distribution system.
- A set of files tracked by the science database.
- A set of files not tracked by the science database.
I. Input to Science Archive
- Survey Definition
- a. A description of the North Imaging survey area
- b. Survey progress: A description of sky inserted into
database to date
- Final Astrometric Calibration
- a. List of r' band calibration coefficients on a
- Final Photometric Calibraton
- a. List of photometric calibration coefficients on a
- Merged Object Lists
- a. A list of calibrated objects and parameters from
the Frames pipeline of photo
- b. A list of objects masks from the Frames pipeline of photo
[TBD: Do a and b provide all information about masked areas of sky?]
- c. Run and Field information. Might be needed for recalibrations)
- d. Star/Galaxy/QSO classifications
- e. Enhanced goal: Cross-identifications to other catalogs
- Target Selection
- a. A list of all targetable objects with target
- b. A list of all objects from a selected as targets
with selection category
- c. Tiling flags for all objects in b.
- d. Reports for all targets selected manually
- Spectroscopic Pipeline
- a. Redshifts and parameters of all targeted objects
- b. Enhanced goal: Tile and plate information.
- Enhanced goal: Scientist derived catalogs
- Enhanced goal: Other input catalogs
- Separate files tracked from Science Database
- a. Atlas Images
- b. 1-D spectra
- Separate files not tracked or accessible from Science Archive
- a. Compressed pixel map
- b. Full corrected pixel map
- c. Corrected spectroscopic frames
- d. Unused data
- Southern Survey:
- a. Same as northern survey.
II. Functional Goals
- User will be able to carry out efficient queries to locate objects
over one or more ranges of following attributes:
- a. Longitude or latitude in several spherical coordinates
- i) J2000 Ra and Dec
- ii) B1950 Ra and Dec (or J1950)
- iii) Enhanced goal: Ra and Dec of arbitrary epoch
- iv) Galactic coordinates
- v) Survey Coordinates
- vi) Any linear combination of the two coordinates
- b. Radius within a given point of the sky
- c. u' g' r' i' z' (One set of magnitudes per object, TBD: which one?)
- d. Any linear combination of c.
- e. Object radius (one per object)
- f. Surface brightess formed by c and d.
- g. Star/Galaxy/QSO Classification flag
- h. Object class (small/medium/big/mask)
- i. Target Selection Category
- j. Spectrum available flag
- User will be able to carry out queries on any retained
object parameter (subject to implementation constraints).
- Enhanced Goal: All calibrated quantities can be recomputed using improved
astrometric and photometric calibrations. Queries can
be performed on the recalibrated quantities.
- For all efficient queries, return an estimated number of objects to be
- For all located objects, users shall be able to specify an arbitrary
subset of stored parameters to be returned (subject to implementation
constraints) plus the following derived quantities:
- a. Number of located objects
- b. TBD: Extra parameters
- Users shall be able to perform the following functions:
- a. Efficient repeated queries [e.g, get all objects
within each of 10,000 QSOs in my favorite catalog).]
- b. Make simple plots, etc. of returned parameters (e.g., SMONGO)
- c. Formulate new queries based on results of previous queries.
- d. TBD: What else?
III. Technical Goals
- User interface
- a. User interface shall be developed in a TCL/Tk/TclDp
- b. User interface shall communicate with a query support layer
via ASCII interface protocol.
- c. Data shall be returnable to files, sockets, or pipes.
Returned data shall use binary machine independent format (FITS binary,
ASCII if appropriate, TBD: FITS ASCII, other?)
- Enhanced goal: Data shall be stored in a system providing industry-standard
OSQL-like interface to enable use of commercial products to provide
alternative view of the database.
- a. A master copy of all data shall be maintained
(the Master Science Archive)
- b. Capability shall be present to replicate all or part of the
Master Science Archive as local databases at SDSS institutions.
Replication may consist of:
- i) Science Database in its entirety
- ii) All or part of separate files tracked by
- iii) No capability shall be present to
replicate an arbitrarily selected
subset of the science database beyond that
described by section 1.c of USER INTERFACE.
- iv) Replication of databases shall be possible
on all SDSS supported platforms.
- d. No capability is required be present to replicate all
or part of separate files not tracked by
- a. Master Science Archive shall be protected against
corruption by SDSS participant users
- b. Master Science Archive shall be protected against
unauthorized access by non-SDSS participants.