Skip to main content

  • If the CSV or delimited format option is specified, the following dataset definition parameters are available:

    • Delimiter—The delimiter used to split field (or column) and record (or row) values. You can choose from the following options or enter your own value:

      • Comma (,)—Field and record values are separated by commas (,). This is the default.

      • Tab (\t)—Field and record values are separated by tabs (\t).

      • Pipe (|)—Field and record values are separated by pipes (|).

      • Semicolon (;)—Field and record values are separated by semicolons (;).

      • Space ( )—Field and record values are separated by spaces ( ).

      If you are entering your own value it must be one or two characters in length, including spaces. Delimiters longer than two characters are not supported.

    • Has header row—Specifies whether the dataset contains a header row. The default is true. If set to false, the first row of the dataset will be considered a record.

    • Has multiline data—Specifies whether the dataset has records that contain new line characters. The default is false. If set to true, data that contains multiline data will be read and formatted correctly.

    • Character encoding—Specifies the encoding type used to read the specified dataset. The default is UTF-8. You can choose from the available encoding options or specify an encoding type. Spaces are not supported in encoding values. For example, specifying a value of ISO 8859-8 is invalid and must be specified as ISO-8859-8.

    • Fields—Optionally configures field names and types. The Configure schema button opens a dialog box containing the dataset fields with the following options:

      • Include or drop fields—You can remove fields by checking the check box next to the field. By default, all fields are included.

      • Field name—The name of the field as it will be used in Data Pipelines. This value can be edited. By default, this value will be the same as the field in the source dataset unless the source name contains invalid characters or is a reserved word. Invalid characters will be replaced with an underscore (_), and reserved words will be prefixed with an underscore (_).

      • Field type—The field type as it will be used in Data Pipelines.

      Removing or modifying fields in Data Pipelines will not modify the source data.

      The following table describes the available field types:

      Field type Description
      String String fields support a string of text characters.
      Small integer Small integer fields support whole numbers between -32768 and 32767.
      Integer Integer fields support whole numbers between -2147483648 and 2147483647.
      Big integer Big integer fields support whole numbers between -9223372036854776000 and 9223372036854776000.
      Float Float fields support fractional numbers between approximately -3.4E38 and 3.4E38.
      Double Double fields support fractional numbers between approximately -2.2E308 and 1.8E308.
      Date Date fields support values in the format yyyy-MM-dd HH:mm:ss, for example, a valid value is 2025-12-31 13:30:30. If the date values are stored in a different format, use the Create date time tool to calculate a date field.
      Date only Date fields support values in the format yyyy-MM-dd, for example, a valid value is 2025-12-31. If the date only values are stored in a different format, use the values as input to the Calculate field tool to calculate a date only field.
      Boolean Boolean fields support values of True and False. If a field contains integer representations of Boolean values (0 and 1), use the Update fields tool to cast the integers to Boolean values instead.
Back to top