Areal Surface Texture Analysis for Both the Lab and the Engineer’s Desk

This tutorial discusses a step-by-step and visual approach to surface texture measurement. The process, which is the basis of the OmniSurf3D software, allows designers, managers, process engineers and lab staff to explore surface texture. OmniSurf3D provides a common language for understanding and communicating surface texture, to help improve component performance.

The article also appeared in March, 2019 on

View the “Areal Surface Texture Analysis” Tutorial.

Controlling Roughness on Short Surfaces

Roughness control is important on many surfaces.  Unfortunately, some of these surfaces can be relatively short – making it difficult to perform “traditional” roughness measurements.


A traditional roughness measurement

A default roughness measurement is based on analyzing 5 sampling lengths (whereby a sampling length is equal to the roughness cutoff wavelength).  See “3 Steps to Understanding Surface Texture” for more on the roughness cutoff wavelength.  These 5 sampling lengths along a surface are referred to as the “evaluation length.”  Thus, for a typical filter cutoff of 0.8 mm, the default evaluation length is 4.0 mm

In an actual measurement there is often the need for the tracing length to be longer than the desired evaluation length.  This is often said to be related to the acceleration and deceleration of the instrument as it starts and stops.  This isn’t necessarily the case with today’s instruments.  With modern instruments, the first data point and the last data point are considered as “good points” and the acceleration/deceleration occurs outside this zone.  With today’s instruments, the extra length at the ends is related to the filtering process.

Let’s consider the Gaussian filter as a weighted moving average.

As can be seen in this (above) figure, the middle of the Gaussian filter does not reach the ends of the profile.   Thus, we need “extra” length on each end of the desired evaluation length.  Some instruments require an extra full cutoff of data on each end of the profile for the Gaussian filter. Other instruments require an extra half cutoff on each end for a Gaussian filter.

For a typical 0.8 mm filter and the desire to achieve a default 4.0 mm evaluation length we have:

For a 1 cutoff discard on each end:           5.6 mm is the required tracing length

For a ½ cutoff discard on each end:          4.8 mm is the required tracing length


What if my surface isn’t that long?

Option 1: Use a different kind of filter

One option is to switch to a filter type that doesn’t require discarding.  In Digital Metrology’s OmniSurf software we have the options of “spline” and “robust” filters.  These filters return the full length of data after filtering… with no discarding at the ends as shown here:

Thus, with OmniSurf’s spline or robust filters, a 4.0 mm evaluation can be achieved with a 4.0 mm tracing length.

In many cases, a spline filter can be used in place of the Gaussian with only minimal differences in computed roughness parameters.  These differences are typically well within the measurement uncertainty of the system.


Option 2: Use fewer sampling lengths

The default evaluation based on 5 sampling lengths is simply that… a default.  We can always use a non-default approach which doesn’t involve 5 sampling lengths.  Ideally this would be noted on the drawing or the measurement report.  If the surface is long enough to support two sampling lengths, then simply report the roughness value for the two available sampling lengths.  If only one sampling length of surface is available, then report the roughness for the one sampling length.


Option 3: Use all available length

OmniSurf’s parameter calculations do not partition the profile into sampling lengths.  Calculations occur over the entire length of filtered data.  (NOTE: OmniSurf does use sampling lengths for RzDIN, RzJIS, Rpm and Rvm as these parameters require partitioning).

Using all available data (particularly with spline or robust filtering) can be a useful approach for short surfaces.  In fact, OmniSurf can compute roughness parameters on less than a full sampling length. This truly means that any length of data can be used with any filter.


Option 4: Use all available length with no filtering

If the available surface length is less than one sampling length, simply levelling the data has nearly the same effect as filtering.  Thus, for a surface length less than one sampling length, we can often replace the roughness “R” parameters (e.g. Ra) with Primary “P” parameters (e.g. Pa).


CAUTION:  Don’t do this…

One (bad) approach for dealing with short surfaces is to choose a shorter filter cutoff to achieve the 5 sampling lengths on the surface.  This can be very dangerous as changing the cutoff wavelength changes the definition of “roughness”.

Let’s look at a small land that has been milled:

The highlighted zone in the above graph is 2.25 mm wide.  This is not enough length for a traditional, 5 sampling length analysis with a typical 0.8 mm cutoff wavelength.  Thus, some people might be inclined to shorten the filter cutoff to achieve 5 sampling lengths.  Here’s what happens on this surface if we shorten the filter cutoff from 0.8 mm down to the next standard cutoff wavelength:  0.25 mm.

This (above) example should be a great reminder that a change in the filter cutoff fundamentally changes the definition of roughness.  The computed “Ra” value is cut in half with the reduction of the cutoff wavelength.

There are many ways to handle short surfaces.  Four good options are presented above.  One thing that should not be done is shorten the filter cutoff.

For more information, contact Digital Metrology today!



Flatness in the USA

Many mechanical interfaces depend on the control of “flatness”.  But what do we really mean by “flatness”?  This is a question that is being raised more and more as tolerances are getting smaller and smaller.  So what do you do when presented with a simple flatness tolerance and a surface to measure?

Let’s start with the definition.  In the US, we have the standard ASME Y14.5 “Dimensioning and Tolerancing.”  At the time of this blog post, the 2018 edition has recently been released.  In it we see:

3.37  Flatness

Flatness is the condition of a surface or derived median plane having all elements in one plane.


But if we are going to measure flatness, we need to understand this definition a bit more. Specifically, “what’s an element?”  Unfortunately, the work “element” is a generic term used throughout Y14.5 to imply anything of interest.  For example, “elements” could be points, circular sections, linear sections, etc. 

In the case of flatness, it makes the most sense from the measurement perspective to consider elements as the data points representing the surface.  Thus, the metrology interpretation of 3.37 would be something like “flatness is the condition of a surface whereby all of the points are in one plane”.


But here’s the big question… ok there are actually two (related) questions?

How many points does it take?   (Sampling)


What kind of filtering/processing do we do with the points?  (Smoothing)


The fundamental challenge with understanding “flatness” boils down to “what do you really mean by flatness?”  For example, this surface can have two, very different flatness values depending on sampling and smoothing:

The reported flatness value depends on the surface wavelengths of interest.  For the above surface, the unfiltered flatness (left) is many times greater than the smoothed flatness (right).  The data on the left side includes all wavelengths.  The data on the right side is smoothed to include only the longer wavelengths.


Does Y14.5 tell us anything about data points and smoothing?

Yes and no. 

Rule #1 (section 5.8.1) famously states that perfect form is required at the maximum material condition.  From the measurement perspective this implies the control of “all unfiltered data points”.  Rule #1 is also referenced in the Y14.5 section 8.2 regarding “form control”.     

Y14.5-2018 is the first revision to provide further clarification on the topic of data points and smoothing.  A new “fundamental rule has been introduced in section 4.2 (note “s”) which states:

Unless otherwise specified (UOS), elements of a surface include surface texture and flaws (e.g., burrs and scratches). All elements of a surface shall be within the applicable specified tolerance zone boundaries.

This new note clarifies what was implied in previous versions of Y14.5; features of form are to include all aspects of surface geometry – regardess of the scale of the feature.

CONCLUSION from Y14.5-2018:  Infinite number of data points.  No smoothing.

Practically speaking, we cannot measure all the way down to the last atom.  Therefore, let’s limit our measurement of flatness to go down to the shortest limit of surface roughness.  A typical roughness measurement is based on using a diamond stylus tip with a 2 micrometer radius (or smaller) and with a data point spacing of no more than 0.5 micrometers.  This can result in an overwhelming number of data points as we consider flatness.  A pencil tip of approximately 0.5mm by 0.5 mm puts this in perspective:


But there’s more to the story…

There is a bit of ambiguity in the US standards as we look a bit deeper.  For example, ASME Y14.5 refers to ASME Y14.5.1-1994 for “Mathematical Definition of Dimensioning and Tolerancing Principles.”   ASME Y14.5.1 has a section (2.1.1) on “Establishing the Surface Points.”  This section indicates that there is certain amount of filtering needed to separate dimensional features from “micro” features like roughness.  The context implies that “dimensional features” are those features defined in Y14.5… which includes flatness.

CONCLUSION from Y14.5.1-1994: Smooth out the roughness.


But wait!  ASME Y14.5.1-1994 also refers to ASME B46.1.  What does B46.1 say on this topic?

Along with the above mentioned note (2.1.1) ASME Y14.5.1 also gives reference to the use of “smoothing functions defined in ASME B46.1”

In the 1990’s ASME B46.1 (v1995 Section 1.2.2) included the concept of “error of Form” as being a type of shape.  It described “errors of form” as being comprised of “widely spaced deviations which are not included in surface texture”.   Since “surface texture” includes roughness and waviness, it implies that “form” is made up of wavelengths longer than waviness.  This results in a “3 domain” definition:


CONCLUSION from ASME B46.1-1995: Smooth out the roughness AND waviness for flatness.


So what is “flatness” in the United States?

In the US we have 3 candidates for flatness per ASME standards:  ASME Y14.5, ASME Y14.5.1, ASME B46.1 (pictured in order left to right)


What do International Standards say about flatness?

In its early drafts, ISO 12781-2 proposed the use of a 0.8 mm cutoff wavelength to remove short wavelengths (i.e. roughness) from flatness.  This 0.8 mm value was chosen since it is the most common surface roughness cutoff wavelength.  Thus, the idea was that there would be two domains:  the roughness domain and the “form” domain.

This approach, whereby roughness is removed from flatness, is much more manageable than trying to use an infinite number of points with no filtering.  However, it can still be difficult to achieve on very large surfaces.  To transmit wavelengths longer than 0.8 mm, a 0.5 mm tip radius is required and the data point spacing must be no greater than 0.15 mm.   Based on the implications of using this on large surfaces, this proposal was rejected by International balloting and the 0.8 mm filter cutoff was removed.  A statement was added to the Standard’s introduction indicating:

At the current state of development, ISO TC 213 has not been able to reach a consensus on defaults for filter UPR, probe tip radius and method of association (reference plane). This means that a flatness specification must explicitly state which values are to be used for these specification operations in order for it to be unique.


CONCLUSION from ISO 12781-2 Early drafts: Smooth out the roughness.

It is important to note that all of the ISO discussions involved the smoothing out of roughness.  The dissention regarding the 0.8 mm filter came from those that were dealing with large surfaces and they actually required more smoothing to extract their shapes of interest.  This ultimately led to the note in the introduction to 12781-2 (2011).

CONCLUSION from ISO 12781-2 (2011): You must specify filtering, sampling, etc.


So… what should you do about this?

First off you need to determine what you mean by flatness.   Are you interested in the general trends of the surface as flatness with aspects being controlled by waviness and/or roughness?  Do you need all unfiltered points captured? 

Ultimately the ISO 12781-2 (2011) standard gives us the best possible direction.  In section 4.1 it tells us that the proper (unambiguous) specification of flatness “defines the transmission band”.  Thus, if you want to control flatness – you need to indicate a wavelength band that describes what you mean by “flatness”.  When presented with a flatness tolerance and a surface to measure, you must ask for more information.  The tolerance limit alone is ambigous.


Still unsure?

For more information and for help specifying and measuring flatness and other surface-related challenges, contact Digital Metrology today!