# precision

(redirected from Double precision)
Also found in: Dictionary, Thesaurus, Legal, Encyclopedia, Wikipedia.

## precision

[pre-sizh´un]
1. the quality of being sharply or exactly defined.
2. in statistics, the extent to which a measurement procedure gives the same results each time it is repeated under identical conditions.

## pre·ci·sion

(prē-si'zhŭn), Do not confuse this word with accuracy.
1. The quality of being sharply defined or stated; one measure of precision is the number of distinguishable alternatives to a measurement.
2. In statistics, the inverse of the variance of a measurement or estimate.
3. Reproducibility of a quantifiable result; an indication of the random error.

## precision

Lab medicine A measure of test or assay reproducibility–ie, capability of producing the same results when performed on the same specimen under the same conditions; data with high precision has a low standard deviation and a low coefficient of variation. Cf Accuracy.

## pre·ci·sion

(prē-sizh'ŭn)
1. The quality of being sharply defined or stated; one measure of precision is the number of distinguishable alternatives to a measurement.
2. statistics The inverse of the variance of a measurement or estimate.
3. Reproducibility of a quantifiable result; an indication of the random error.

## precision

the quality of being sharply defined by virtue of exact detail, an important criteria of a diagnostic test. A precise test is free from random error. Precision is a requirement of accuracy.
References in periodicals archive ?
Approximately 1-5 % performance gain is obtained with double precision.
Caption: FIGURE 5: Performance comparison for double precision on C2050.
Finally, due to the very high accuracy achieved by the approximate prolate and Knab window as well as the NUFFT, the minimum achieved coherence values have been, for the double precision case, equal to 1 in all the three cases.
This research implemented a simple conversion method for converting 16bit fixed point numbers to 64bit IEEE 754 double precision floating point number and vice versa.
The details of test problems in double precision are given in Table 1.
To mimic pure single precision and double precision in the absence of overflow and underflow, one has to set the rounding precision (called precision control) to 24, respectively 53 bits.
If speed is not important, one may program a congruential generator in double precision.
The results in Section 5 of this article show that the new exponential and Cauchy sampling routines are about as fast as a square root--in standard and double precision.
With nearly 38 times the performance of the closest competing product based on double precision testing1, and powered by OpenCL , the AMD FirePro W8100 is designed for the next generation of 4K CAD (Computing Aided Design) and Media and Entertainment (M&E) workflows, engineering analysis and supercomputing applications.
where prec is the number of bytes corresponding to the precision used (four bytes for single precision and eight bytes for double precision in IEEE format), while CS is the cache size in bytes.
Intel's "Knights Corner" product, the first commercial co-processor based on the Intel[R] Many Integrated Core (Intel[R] MIC) architecture, was shown for the first time breaking the barrier of 1 TFLOPS double precision performance**.

Site: Follow: Share:
Open / Close