Datafoundry fixed-width ingestion provides an option to append new columns which are extracted from a "match regex" and "extract regex" applied from the filename.


As an example, we are appending a new column during file ingestion based on the following format of the filename as 'file_YYMMDD-HHMMSS'


Step 1:  Configure a file ingestion source and crawl for metadata

              Ref: https://docs2x.infoworks.io/data-ingestion/file-ingestion-process


Step 2: Click on 'Configure', click 'Edit Schema' tab, click on add column and check on 'Extract from filename'


Step 3: Name the column as you wish, select column as 'timestamp', for option specify the format for the timestamp.

              Example: yyyy-MM-dd hh:mm:ss


Step 4: Fill in the regular expression and extract format as below for a file name with the format as 

          'file_YYMMDD-HHMMSS'


               For Regex : (\d\d\d\d)(0[1-9]|1[0-9])(0[1-9]|[12][0-9]|3[01])(-)(0[0-9]|[0-9])(0[0-9]|[0-9])(0[0-9]|[0-9])
               For Extract Format: $1-$2-$3 $5:$6:$7

Step 5: Click Save Schema 


Step 6: Initialize and Ingest 



Applicable Infoworks DataFoundry Versions:

v.2.x, 3.x