Basic Usage

#2021/04/05 Tutorial on basic functions
#
# these lines update the imported module if code is changed, you can remove them if not needed.

%load_ext autoreload
%autoreload 2

The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload

%reset
import os
import numpy as np

Overview

PySurf library consists in a set of classes and functions, representing 2D (or related) data and operations on them.

First thing is importing it:

from pySurf.data2D_class import Data2D

<frozen importlib._bootstrap>:228: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 96 from PyObject

The most basic way to initialize such an object is by passing directly 2D data, (optionally) coordinates and options.

Come vediamo puo’ essere inizializzato con data,x,y e tante altre cose. Tipicamente pero’ vorremo leggere dati da qualche file. C’e’ un’argomento reader per selezionare una funzione per leggere i dati, ve ne sono per molti comuni strumenti e formati, o puo’ essere implementata custom. Se il reader non e’ passato esplicitamente, prova ad indovinare il formato, ed in genere ci azzecca abbastanza.

Importing Data

Quindi proviamo a leggere qualche dato (salvato da MFT in formato testo):

Functions for reading common formats of 2D data are collected in pySurf.readers module. The structure and interface of readers is described elsewhere, a reader is essentially a function able to obtain data, x, y from a data file, however if the interface is correctly implemented, a reader from pySurf.readers.instrumentReader can be passed as argument to at object creation. In this case, additional information (e.g. from header or metadata) are automatically added to the object.

For example, here we read an AFM file in .nid format:

from pySurf.data2D_class import Data2D
from pySurf.instrumentReader import nid_reader

D = Data2D(file, strip=True, reader = nid_reader)
D.plot()

module moved, modify your import to use readers.instrumentReader

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
c:\Users\kovor\Documents\python\pyXTel\docs\source\notebooks\basic_usage.ipynb Cell 12 line 4
      <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X14sZmlsZQ%3D%3D?line=0'>1</a> from pySurf.data2D_class import Data2D
      <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X14sZmlsZQ%3D%3D?line=1'>2</a> from pySurf.instrumentReader import nid_reader
----> <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X14sZmlsZQ%3D%3D?line=3'>4</a> D = Data2D(file, strip=True, reader = nid_reader)
      <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X14sZmlsZQ%3D%3D?line=4'>5</a> D.plot()

NameError: name 'file' is not defined

d.plot()

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
c:\Users\kovor\Documents\python\pyXTel\docs\source\notebooks\basic_usage.ipynb Cell 13 line 1
----> <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X15sZmlsZQ%3D%3D?line=0'>1</a> d.plot()

NameError: name 'd' is not defined

if no options are provided, the library tries to guess from file extension, and in general tries to complete the object with all possible information from data and metadata. Here a file is read from a standard Zygo .dat format:

fn2 = r'..\..\..\source\pySurf\test\input_data\MFT\08_cos02_bare.dat'

d = Data2D(fn2)
print ("Return object", d)

d.plot()

  Cell In[9], line 1
    if no options are provided, the library tries to guess from file extension, and in general tries to complete the object with all possible information from data and metadata. Here a file is read from a standard Zygo `.dat` format.
          ^
SyntaxError: invalid syntax

Notiamo tuttavia che le unita’ degli assi sono scomode (ed ignote), vorrei anche collocare il centro dell’immagine sull’origine (potrebbe pero’ essere ovunque). I can also invert y axis to match format specs:

d = Data2D(fn2,units=['mm','mm','nm'],center=(0,0),scale=(1000,-1000,1))  #this matches exactly Gwyddion
d.plot()

first argument is string, use it as filename

c:\users\kovor\documents\python\pyxtel\source\pySurf\data2D.py:1301: DeprecationWarning: invalid escape sequence \m
  '''
c:\users\kovor\documents\python\pyxtel\source\pySurf\readers\format_reader.py:54: DeprecationWarning: invalid escape sequence \M
  '''from manual: Complete maps of the header formats can be obtained by running the dat_test.exe

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
c:\Users\kovor\Documents\python\pyXTel\docs\source\notebooks\basic_usage.ipynb Cell 16 line 1
----> <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X21sZmlsZQ%3D%3D?line=0'>1</a> d = Data2D(fn2,units=['mm','mm','nm'],center=(0,0),scale=(1000,-1000,1))  #this matches exactly Gwyddion
      <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X21sZmlsZQ%3D%3D?line=1'>2</a> d.plot()

File c:\users\kovor\documents\python\pyxtel\source\pySurf\data2D_class.py:243, in Data2D.__init__(self, data, x, y, file, reader, units, name, *args, **kwargs)
    241     reader = auto_reader(file)  # returns a reader
    242 # calling without arguments skips register, however skips also reader argumnets, temporarily arranged with pop in read_data to strip all arguments for
--> 243 data, x, y = read_data(file, reader, *args, **kwargs)
    244 # register data and pass the rest to reader
    245 # pdb.set_trace()
    247 if np.size(x) == data.shape[1]:

File c:\users\kovor\documents\python\pyxtel\source\pySurf\data2D.py:549, in read_data(file, rreader, **kwargs)
    541 strip=kwargs.pop('strip',False)
    542 ##regdic={'scale':scale,'crop':crop,'center':center,'strip':strip}
    543
    544 #kwargs=pop_kw(kwargs,['scale','crop','center','strip'],
   (...)
    547 #get_data using format_reader
    548 #pdb.set_trace()
--> 549 data,x,y=rreader(file,**kwargs)
    551 return register_data(data,x,y,scale=scale,crop=crop,
    552     center=center,strip=strip,**kwargs)

File c:\users\kovor\documents\python\pyxtel\source\pySurf\readers\format_reader.py:94, in csv4D_reader(wfile, ypix, ytox, header, delimiter, endline, skip_header, *args, **kwargs)
     84 def csv4D_reader(wfile,ypix=None,ytox=None,header=False,delimiter=',',endline=True,skip_header=12,*args,**kwargs):
     85     """read csv data in 4sight 4D format.
     86     12 lines header with info in namelist format, uses `xpix`, `aspect` and `wavelength` if available.
     87     Note that standard csv format ends line with `,` which adds an extra column.
   (...)
     91     2020/07/14 read data directly with `np.genfromtxt`,
     92     rather than uselessely launching the wrapper `data2D.data_from_txt`."""
---> 94     head=read_pars_from_namelist(wfile,': ') #this returns a dictionary, order is lost if header is returned.
     95     if header:
     96         return '\n'.join([": ".join((k,v)) for (k,v) in head.items()])+'\n'

File c:\users\kovor\documents\python\pyxtel\source\dataIO\read_pars_from_namelist.py:19, in read_pars_from_namelist(filename, separator)
     17     l=filename.readlines()
     18 else:
---> 19     l=open(filename,'r').readlines()
     21 return namelist_from_string(l,separator=separator)

FileNotFoundError: [Errno 2] No such file or directory: 'G:\\My Drive\\progetti\\c_overcoating\\esperimenti\\20200214_batch2_IrC\\20200306_MFT_calibration\\01_18803_A.csv'

This is an example of how I can read a simple matrix csv file, passed by a colleague, that was stripped of the header and has an arbitrary delimiter, scale (correct by wavelength), etc.

fn = r'..\..\..\source\pySurf\test\input_data\MFT-txt\EN4-2-100.txt'
d2 = Data2D(fn,units=['mm','mm','nm'],center=(0,0),matrix=1,delimiter='',scale=(0.001,0.001,635.)) #this matches exactly MFT software
d2.plot()

first argument is string, use it as filename
fileformat ``.txt``not recognized for file ..\..\..\source\pySurf\test\input_data\MFT-txt\EN4-2-100.txt
Use generic text reader

<Axes: title={'center': 'EN4-2-100.txt'}, xlabel='X (mm)', ylabel='Y (mm)'>

The returned object represents a set of data and axis. These can be returned calling the object itself:

d()

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
c:\Users\kovor\Documents\python\pyXTel\docs\source\notebooks\basic_usage.ipynb Cell 20 line 1
----> <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X25sZmlsZQ%3D%3D?line=0'>1</a> d()

NameError: name 'd' is not defined

Data manipulation functions

Basic operations like cropping or leveling can be applied by means of corresponding methods. Information about methods and options can be obtained by means of usual Python introspection methods.

Here some example:

D2 = D.level((4,2))
D2.plot()  #Level 4 legendre along `x` and 2 along `y`.

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
c:\Users\kovor\Documents\python\pyXTel\docs\source\notebooks\basic_usage.ipynb Cell 22 line 1
----> <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X30sZmlsZQ%3D%3D?line=0'>1</a> D2 = D.level((4,2))
      <a href='vscode-notebook-cell:/c%3A/Users/kovor/Documents/python/pyXTel/docs/source/notebooks/basic_usage.ipynb#X30sZmlsZQ%3D%3D?line=1'>2</a> D2.plot()  #Level 4 legendre along `x` and 2 along `y`.

NameError: name 'D' is not defined

plotting module contains commodity functions for plotting of data and comparisons, we use plotting.multiplots.compare_images to compare the original data with the modified version.