DataScope Select - REST API

Close Menu
Expand All Collapse All
Introductory Tutorials Tutorials Introduction Programming without SDK Tutorial REST API Tutorials REST API Tutorials Introduction REST API Tutorial 1: Connecting to the DSS server REST API Tutorial 2: On Demand End of Day Extraction REST API Tutorial 3: On Demand intraday extraction, embargo REST API Tutorial 4: On Demand price history extraction REST API Tutorial 5: On Demand corporate actions extraction REST API Tutorial 6: On Demand ownership data extraction REST API Tutorial 7: On Demand T&C extraction REST API Tutorial 8: On Demand composite extraction REST API Tutorial 9: On Demand extraction: instrument list REST API Tutorial 10: GUI control calls: immediate extract REST API Tutorial 11: Search by Instrument REST API Tutorial 12: Search for an Equity REST API Tutorial 13: Search for a Future or Option REST API Tutorial 14: On Demand price history extraction raw .Net SDK Tutorials .Net SDK Tutorial 1: Connecting to the DSS server .Net SDK Tutorial 2: GUI control calls: List, report, sched .Net SDK Tutorial 3: GUI control calls: Validate, extraction .Net SDK Tutorial 4: GUI control calls: Embargo, note files .Net SDK Tutorial 5: On Demand: EoD extraction .Net SDK Tutorial 6: On Demand: EoD extraction, file I/O .Net SDK Tutorial 7: On Demand: large instrument lists .Net SDK Tutorial 8: On Demand: Terms & Conditions .Net SDK Tutorial 9: On Demand: Composite extraction .Net SDK Tutorial 10: Search by Instrument .Net SDK Tutorial 11: Search for Equity .Net SDK Tutorial 12: Search for Future or Option

.Net SDK Tutorial 6: On Demand: EoD extraction, file I/O

Last update July 2019
Environment Windows
Language C#
Compilers Microsoft Visual Studio 2012/2013
Prerequisites DSS login, internet access, having done the previous tutorials
Source code Download .Net SDK Tutorials Code

Tutorial purpose

This is the sixth tutorial in a series of .Net SDK tutorials. It is assumed that the reader has acquired the knowledge delivered in the previous tutorials before following this one.

In this tutorial we build on the previous one by adding file input and output, and some error handling:

  • Populate a large instrument list array read from a CSV file, and log input file errors to a file.
  • Write the received data and extraction notes (including maintenance notes) to files.
  • Analyze the extraction notes.
  • This sample contains 2 extraction requests: one with the ExtractWithNotes endpoint, the other with the ExtractRaw endpoint, to illustrate the differences between the two.

The first 3 added capabilities do not illustrate additional DSS REST API functionality, but serve to put it in the context of productized code. The last one illustrates an API capability that can be useful, depending on the use case.

 

Table of contents

 

Getting ready

Opening the solution

The code installation was done in Tutorial 1.

Opening the solution is similar to what was done in the previous tutorials:

  • Navigate to the \DSS REST API\Tutorial 6\Learning folder.
  • Double click on the solution file rest_api_more_advanced_topics.sln to open it in Microsoft Visual Studio.

 

Referencing the DSS SDK

Before anything else, you must reference the DSS REST API .Net SDK in the Microsoft Visual Studio project.

Important: this must be done for every single tutorial, for both the learning and refactored versions.

This was explained in the tutorial 2; please refer to it for instructions.

 

Viewing the C# code

In Microsoft Visual Studio, in the Solution Explorer, double click on Program.cs and on DssClient.cs to display both file contents. Each file will be displayed in a separate tab.

 

Setting the user account

Before running the code, you must replace  YourUserId  with your DSS user name, and  YourPassword  with your DSS password, in these 2 lines of Program.cs:

        private static string dssUserName = "YourUserId";
        private static string dssUserPassword = "YourPassword";

Important reminder: this must be done for every single tutorial, for both the learning and refactored versions.

Failure to do so will result in an error at run time (see Tutorial 1 for more details).

 

Important warning on data usage

DSS data request quota

DSS accounts have a quota of instruments that can be requested during a defined time period. The DSS subscription fees are proportional to the size of the quota. If the quota is reached, excess fees will be due. That situation must be avoided !

Data usage can be displayed in the DSS web GUI:

Contact your local account manager or sales specialist if you have queries on your DSS account quota or usage.

 

Instrument list files delivered with the tutorials

This tutorial (and the next one) handle large instrument lists, read from a file.

Care must be exercised when running these tutorials, to avoid over consuming data.

For this reason, several files are delivered with the tutorials code:

  • DSS_API_1500_input_file.csv
  • DSS_API_60_input_file.csv
  • DSS_API_10_input_file.csv

The first one contains 1500 valid instrument identifiers, the second one 60, the last one only contains 10.

The text and illustrations that follow use the larger file. To avoid consuming too much data, we recommend you use the smaller file for your own tests, which will deliver similar results.

 

Understanding the code

We shall only describe what is new versus the previous tutorials.

DssClient.cs

This is the same as the refactored version of Tutorial 5, except for the leading comment, and an additional method at the end to create an EoD extraction that delivers compressed data, which we describe further down.

No additional explanations are required as the rest of the code was described in the previous tutorial.

 

Program.cs

At the top of the code we see a using directive was added for file input / output:

using System.IO;

 

Member declarations

We add a whole set of declarations, to define the input and output files and their locations. As output we will create a data file, a notes file and a separate error log file. As input we have an instrument identifier list:

private static string outputDirectory = "C:\\DSS REST API\\";
private static string dataOutputFile = outputDirectory + "DSS_API_tutorial_6l_output.txt";
private static string notesOutputFile = outputDirectory + "DSS_API_tutorial_6l_notes.txt";
private static string errorOutputFile = outputDirectory + "DSS_API_tutorial_6l_errors.txt";
private static string gzipDataOutputFile = outputDirectory + "DSS_API_tutorial_6l_gzipOutput.csv.gz";
private static string gzipNotesOutputFile = outputDirectory + "DSS_API_tutorial_6l_gzipNotes.txt";

private static string inputDirectory = outputDirectory;
private static string instrumentIdentifiersInputFile = inputDirectory + "DSS_API_10_input_file.csv";

The default directories and file names above can be used as is if you installed the tutorials code as described in Tutorial 1, using the directory structure defined in the Quick Start. If you are using a different folder structure, change the declarations accordingly.

The input file is a CSV file. A sample is included with the code, but you can also make your own.

  • File format: 1 instrument per line.
  • Line format: <identifier type>, <code>, <optional comment>

Example lines, first two are for a Cusip (CSP) and a Reuters Instrument Code (RIC):

CSP,31352JRV1
RIC,ALVG.DE
CHR,0#.FCHI,CAC40
CIN,G93882192,Vodaphone
COM,001179594,Carrefour
ISN,CH0012138530,CS
SED,B1YW440,3i Group
VAL,24476758,UBS
WPK,A11Q05,Elumeo

The sample input files contains lines with correct line format syntax. They also include lines that are comments, empty or badly structured, to test the error handling included in the code.

To avoid consuming too much data, we recommend you use small files for your own tests. For more information, refer to the previous section: Important warning on data usage.

 

Initial file and directory management

We start with some basic input checks.

First we verify if the input file exists. If it does not, we display an error and exit the program:

if (!File.Exists(instrumentIdentifiersInputFile))
{
    DebugPrintAndWaitForEnter("ERROR accessing " + instrumentIdentifiersInputFile +
        "\nCheck if file and directory exist.");
    return;  //Exit main program
}

 

Then we verify if the output directory exists. If it does not, we display an error and exit the program:

if (!Directory.Exists(outputDirectory))
{
    DebugPrintAndWaitForEnter("ERROR accessing directory " + outputDirectory + "\nCheck if it exists.");
    return;  //Exit main program
}

 

Then we clear the output files of any content left over from a previous run, and create them if they do not exist. Here is the code for one of the files (they all use the same code):

StreamWriter sw = new StreamWriter(dataOutputFile, false);
sw.Close();
Console.WriteLine("Cleared data output file " + dataOutputFile + "\n");

The second parameter of StreamWriter defines if content is to be appended to the file. By setting a value of false we clear the contents if the file already exists.

 

Next, we open the error output file and initialise it with a first message:

//Initialise the error output file:
sw = new StreamWriter(errorOutputFile, true);
sw.WriteLine("List of errors found in input file: " + instrumentIdentifiersInputFile + "\n");

We also open the input file:

StreamReader sr = new StreamReader(instrumentIdentifiersInputFile);

 

Creating the instrument list by reading an input file

Instead of creating an array of defined length like in the previous tutorial, we shall create a list for the instrument identifiers.

The reason is that, at this stage, we do not know how many instrument identifiers we will have, because:

  • We don't know how many instruments are in the file.
  • We shall validate the file entries, and only keep the good ones.

It is not possible to define an array of undefined size, and it would be a waste to define a huge one and then resize it.

So we define a list of undefined size, which we shall fill from the file. Once all instrument identifiers have been added to the list, we convert it to an array, as that is what the On Demand extraction API call requires.

The first step is to create the empty instrument identifiers list:

List<InstrumentIdentifier> instrumentIdentifiersList = new List<InstrumentIdentifier> ();

Then we populate the list, by reading one line at a time, from the input file. First we initialize some variables:

int fileLineNumber = 0;
string fileLine = string.Empty;
string identifierTypeString = string.Empty;
string identifierCodeString = string.Empty;
IdentifierType identifierType;
int i = 0;

Then we read all lines until we get to the end of the file, parse the data, and add the validated data to the list:

bool endOfFile = false;
while (!endOfFile)
{
    //Read one line of the file, test if end of file:
    fileLine = sr.ReadLine();
    endOfFile = (fileLine == null);
    if (!endOfFile)
    {
        fileLineNumber++;
        try
        {
            //Parse the file line to extract the comma separated instrument type and code:
            string[] splitLine = fileLine.Split(new char[] { ',' });
            identifierTypeString = splitLine[0];
            identifierCodeString = splitLine[1];
            //Add only validated instrument identifiers into our list.
            if (identifierTypeString != string.Empty)
            {
                if (identifierCodeString != string.Empty)
                {
                    //DSS can handle many types, here we only handle a subset:
                    switch (identifierTypeString)
                    {
                        case "CHR": identifierType = IdentifierType.ChainRIC; break;
                        case "CIN": identifierType = IdentifierType.Cin; break;
                        case "COM": identifierType = IdentifierType.CommonCode; break;
                        case "CSP": identifierType = IdentifierType.Cusip; break;
                        case "ISN": identifierType = IdentifierType.Isin; break;
                        case "RIC": identifierType = IdentifierType.Ric; break;
                        case "SED": identifierType = IdentifierType.Sedol; break;
                        case "VAL": identifierType = IdentifierType.Valoren; break;
                        case "WPK": identifierType = IdentifierType.Wertpapier; break;
                        default:
                            identifierType = IdentifierType.NONE;
                            DebugPrintAndWriteToFile("ERROR line " + fileLineNumber +
                                ": unknown identifier type: " + identifierTypeString, sw);
                            break;
                    }
                    if (identifierType != IdentifierType.NONE)
                    {
                        instrumentIdentifiersList.Add(new InstrumentIdentifier
                        {
                            IdentifierType = identifierType,
                            Identifier = identifierCodeString
                        });
                        Console.WriteLine("Line " + fileLineNumber + ": " + identifierTypeString +
                            " " + identifierCodeString + " loaded into array [" + i + "]");
                        i++;
                    }
                }

Note: for more information on identifier types, refer to Tutorial 2: Creating an instrument list.

The code that follows is for error logging. Each error is displayed on screen and written to file, using a helper method:

                //Error handling messages grouped here:
                else
                {
                    DebugPrintAndWriteToFile("ERROR line " + fileLineNumber +
                        ": missing identifier code in line: " + fileLine, sw);
                }
            }
            else
            {
                DebugPrintAndWriteToFile("ERROR line " + fileLineNumber +
                    ": missing identifier type in line: " + fileLine, sw);
            }
        }
        catch
        {
            DebugPrintAndWriteToFile("ERROR line " + fileLineNumber +
                ": bad line format: " + fileLine, sw);
        }
    }
    else
    {
        if (fileLineNumber == 0)
            { DebugPrintAndWriteToFile("ERROR: empty file: " + instrumentIdentifiersInputFile, sw); }
    }
}  //End of while loop

 

The helper method which displays errors and writes them to file is defined at the end of the main code:

static void DebugPrintAndWriteToFile(string messageToPrintAndWriteToFile, StreamWriter sw)
{
    Console.WriteLine(messageToPrintAndWriteToFile);
    sw.WriteLine(messageToPrintAndWriteToFile);
}

 

Extract of the output for successfully added instrument identifiers (using the largest sample input file):

Extract of the output for invalid lines in the largest input file:

The error file contains all the ERROR messages that appear on screen, in the same format.

 

Back to the main code, the next step is to close the input file and the error output file:

sr.Close();
sw.Close();

 

This is followed with a check on the number of valid instrument identifiers:

int validIdentifiersCount = i;
if (validIdentifiersCount == 0)
{
    DebugPrintAndWaitForEnter("Exit program due to no identifiers in the list.");
    return;  //Exit main program
}
Console.WriteLine("\n" + validIdentifiersCount +
    " valid instruments were loaded into an array, outside of DSS,\n" +
    "for use in the extraction.");

Result:

 

Now we convert our instrument identifiers list to an array, as that is what the On Demand extraction API call requires:

//Convert the instrument identifiers list to an array:
InstrumentIdentifier[] instrumentIdentifiers = instrumentIdentifiersList.ToArray();

 

Creating the field name array

To create the field name array we proceed just like in the previous tutorials, calling the helper method we created in Tutorial 2:

string[] requestedFieldNames = CreateRequestedFieldNames();

We will use this array when we define the extraction.

 

No report template or schedule creation

Like in the previous tutorial, we do not create a:

  • Report template, because the On Demand extraction calls implicitly define it.
  • Schedule, because an On Demand extraction is on the fly.

 

Creating and running an On Demand extraction (for JSON data)

This is what we did in the previous tutorial. The extraction uses the ExtractWithNotes endpoint, which delivers the data in JSON format.

We call the helper method we created in the previous tutorial:

ExtractionResult extractionResult =
    dssClient.CreateAndRunEodPricingExtraction(instrumentIdentifiers, requestedFieldNames);
DssCollection<ExtractionRow> extractionDataRows = extractionResult.Contents;

 

Retrieving the extracted data and notes, writing them to file

We call three helper methods:

DisplayAndLogExtractedDataFieldNames(dataOutputFile, extractionDataRows);
DisplayAndLogExtractedDataFieldValues(dataOutputFile, extractionDataRows);
DisplayAndLogAndAnalyzeExtractionNotes(notesOutputFile, errorOutputFile, extractionResult);

These methods are small variations of those we created in the previous tutorial to retrieve and display the data. They format the data as comma separated values (CSV).

The first difference with the previous tutorial lies in the fact that these will also write to file, so they have an additional input parameter (the data output file name), and 3 additional code lines, one to open the output file:

StreamWriter sw = new StreamWriter(outputFile, true);

One to write data to the file:

sw.WriteLine(returnedFieldNames.ToString());

And one to close the file afterwards:

sw.Close();

 

The second difference is that the extraction notes are parsed to detect if the extraction was successful, and if there were any warnings, errors or permission issues. This is based on simple string content analysis.

Here are 2 extracts of the results on screen, first the extracted field names:

These are some of the (many) data values:

The data output file contains all the data lines, in exactly the same format, preceded with the list of field names.

 

Here we have 1500 instrument identifiers in the file, 1486 returned data. The missing 14 were not quoted on the day of the request, either because they are not volatile or maybe they are expired (like expired bonds or futures):

 

The extraction notes are displayed and saved to file. Their analysis results are also displayed.

 

Creating and running an On Demand extraction (for compressed CSV data)

This is a different possibility we illustrate in this tutorial. Instead of retrieving JSON formatted data we want compressed CSV data, which is practical for storage. This also optimises extractions of large data sets.

Instead of the ExtractWithNotes endpoint (which delivers the data in JSON format), the extraction uses the ExtractRaw endpoint, which delivers compressed CSV formatted data.

Depending on your use case, you can choose either ExtractWithNotes or ExtractRaw.

We call a new helper method, that is very similar to the one we used previously:

RawExtractionResult rawExtractionResult =
    dssClient.CreateAndRunEodPricingRawExtraction(instrumentIdentifiers, requestedFieldNames);

The difference between this helper method and the previous one is the endpoint, and the fact that it returns a raw extraction result. This also implies that we do not extract data rows from the extraction result contents.

 

Retrieving the compressed data, writing it to file

We must tell the server that we accept compressed data, by setting the Accept-Encoding header. If we do not do this, the server will send uncompressed data.

We then use a stream to retrieve the data and write it to a Gzipped file, which contains the data in CSV format:

extractionsContext.DefaultRequestHeaders.Add("Accept-Encoding", "gzip");
DssStreamResponse streamResponse = extractionsContext.GetReadStream(rawExtractionResult);
using (FileStream fileStream = File.Create(gzipDataOutputFile))
    streamResponse.Stream.CopyTo(fileStream);
extractionsContext.DefaultRequestHeaders.Remove("Accept-Encoding");

 

Retrieving the extraction notes for the compressed data, writing them to file

The notes are available directly in the raw extraction result. We call a new helper method, which is practically the same as the one we used for the uncompressed data. The only difference is the 3rd input, which is a raw extraction result:

DisplayAndLogAndAnalyzeRawExtractionNotes(gzipNotesOutputFile, errorOutputFile, rawExtractionResult);

 

No cleaning up

As stated previously, cleanup on the DSS server is not required, as there is nothing to delete in this case.

 

Full code

The full code can be displayed by opening the appropriate solution file in Microsoft Visual Studio.

 

Summary

List of the main steps in the code:

  1. Authenticate by creating an extraction context.
  2. Check input file and output directory existence, clear output files.
  3. Create an array of financial instrument identifiers, populate it from a file. Manage and log errors.
  4. Create an array of field names.
  5. Create an extraction.
  6. Run the extraction, wait for it to complete. Retrieve the extracted data and extraction notes, display them and write them to files.
  7. Variant of step 6, for compressed data.

We do not create a report template or schedule, and there is no need to cleanup.

 

Code run results

Build and run

Don’t forget to reference the DSS SDK, and to set your user account in Program.cs !

 

Successful run

After running the program, and pressing the Enter key when prompted, the final result (using the largest input file) should look like this:

This goes on with the next instruments of the file ...

The instrument identifiers import messages end like this, with quite a few errors generated by the wrongly formatted lines we included in the input file to test the error handling:

This is followed by the extraction, and the display of the returned field names and instrument data:

This goes on with all the returned data, followed by the final statistics and messages:

The extraction notes and analysis are also displayed:

We then run the second extraction, using the ExtractRaw endpoint, to retrieve compressed data. We save the file, then display and analyze the extraction notes:

We find the output files here; their contents reflect what was displayed on screen:

 

Potential errors and solutions

If the user name and password were not set properly, an error will be returned. See Tutorial 1 for details.

 

Understanding the refactored version

Explanations

DSS client helper class file: DssClient.cs

This has not changed.

 

Main program file: Program.cs

We enhance the error reporting. In the member declaration section, we add a list of possible errors, which we will use in input file error handling and reporting:

private enum Errors
    { NoError, EmptyFile, BadLineFormat, EmptyType, EmptyCode, UnknownIdentifier };

 

To check if the input file exists, we call a helper method:

if (!FileExists(instrumentIdentifiersInputFile)) { return; }  //Exit main program

This helper method is declared in Program.cs, after the main code:

static bool FileExists(string fileName)
{
    if (!File.Exists(fileName))
    {
        DebugPrintAndWaitForEnter("ERROR accessing " + fileName +
            "\nCheck if file and directory exist.");
        return false;
    }
    else { return true; }
}

 

To check if the output file exists, we call another helper method:

if (!DirectoryExists(outputDirectory)) { return; }  //Exit main program

This helper method is declared in Program.cs, after the main code:

static bool DirectoryExists(string dirName)
{
    if (!Directory.Exists(dirName))
    {
        DebugPrintAndWaitForEnter("ERROR accessing directory " + dirName + "\nCheck if it exists.");
        return false;
    }
    else { return true; }
}

 

To clear the contents of the output files (if they exist), we call a helper method. Here is the code for one of the files (they all use the same code):

ClearFileContents(dataOutputFile);
Console.WriteLine("Cleared data output file " + dataOutputFile + "\n");

This helper method is declared in Program.cs, after the main code:

static void ClearFileContents(string fileName)
{
    StreamWriter sw = new StreamWriter(fileName, false);
    sw.Close();
}

 

To create the array of instrument identifiers, and populate it from the input file, we call a helper method:

InstrumentIdentifier[] instrumentIdentifiers =
    PopulateInstrumentIdentifiersFromFile(instrumentIdentifiersInputFile, errorOutputFile);

This helper method is declared in Program.cs, after the main code. Note how error handling was restructured to use variable errorCode, to which we assign one of the values of the list we declared in the member declarations: 

static InstrumentIdentifier[] PopulateInstrumentIdentifiersFromFile(
    string instrumentIdentifiersInputFile, string errorOutputFile)
{
    //Open the input file:
    StreamReader sr = new StreamReader(instrumentIdentifiersInputFile);

    //Initialise the error output file:
    StreamWriter sw = new StreamWriter(errorOutputFile, true);
    sw.WriteLine("List of errors found in input file: " + instrumentIdentifiersInputFile + "\n");

    //Instead of an array (of defined length), create a list for the instrument identifiers.
    //We do this because:
    //  we don't know how many instruments are in the file,
    //  we filter the file entries, to only keep the validated ones.
    //Once the list is filled, we convert it to an array, as that is what the API needs.
    //Create the empty instrument identifiers list:
    List<InstrumentIdentifier> instrumentIdentifiersList = new List<InstrumentIdentifier> ();

    //Populate the list, reading one line at a time from the file:
    int fileLineNumber = 0;
    string fileLine = string.Empty;
    bool commaExistsInFileLine = false;
    string identifierTypeString = string.Empty;
    string identifierCodeString = string.Empty;
    IdentifierType identifierType;
    int i = 0;

    //Loop through all lines until we get to the end of the file:
    bool endOfFile = false;
    while (!endOfFile)
    {
        Errors errorCode = Errors.NoError;

        //Read one line of the file, test if end of file:
        fileLine = sr.ReadLine();
        endOfFile = (fileLine == null);
        if (endOfFile && fileLineNumber == 0) { errorCode = Errors.EmptyFile; };
        fileLineNumber++;

        //Parse the file line to extract the comma separated instrument type and code:
        if (errorCode == Errors.NoError && !endOfFile)
        {
            commaExistsInFileLine = (fileLine.IndexOf(",") >= 0);
            if (commaExistsInFileLine)
            {
                string[] splitLine = fileLine.Split(new char[] { ',' });
                identifierTypeString = splitLine[0];
                identifierCodeString = splitLine[1];
            }
            else
            {
                errorCode = Errors.BadLineFormat;  //Missing comma
                identifierTypeString = string.Empty;  
                identifierCodeString = string.Empty;
            }
        }
        if (identifierTypeString == string.Empty && errorCode == Errors.NoError)
            { errorCode = Errors.EmptyType; }
        if (identifierCodeString == string.Empty && errorCode == Errors.NoError)
            { errorCode = Errors.EmptyCode; }

        identifierType = IdentifierType.NONE;
        if (errorCode == Errors.NoError && !endOfFile)
        {
            //DSS can handle many types, here we only handle a subset:
            switch (identifierTypeString)
            {
                case "CHR": identifierType = IdentifierType.ChainRIC; break;
                case "CIN": identifierType = IdentifierType.Cin; break;
                case "COM": identifierType = IdentifierType.CommonCode; break;
                case "CSP": identifierType = IdentifierType.Cusip; break;
                case "ISN": identifierType = IdentifierType.Isin; break;
                case "RIC": identifierType = IdentifierType.Ric; break;
                case "SED": identifierType = IdentifierType.Sedol; break;
                case "VAL": identifierType = IdentifierType.Valoren; break;
                case "WPK": identifierType = IdentifierType.Wertpapier; break;
                default: errorCode = Errors.UnknownIdentifier; break;
            }
        }

        if (errorCode == Errors.NoError && !endOfFile)
        {
            //Add validated instrument identifier into our list:
            instrumentIdentifiersList.Add(new InstrumentIdentifier
            {
                IdentifierType = identifierType,
                Identifier = identifierCodeString
            });
            Console.WriteLine("Line " + fileLineNumber + ": " +
                identifierTypeString + " " + identifierCodeString + " loaded into array [" + i + "]");
            i++;
        }

        if (errorCode != Errors.NoError)
        {
            DebugPrintAndWriteToFileErrorMessage(
                errorCode, fileLineNumber, fileLine, identifierTypeString, sw);
        }
    }  //End of while loop

    sr.Close();
    sw.Close();

    //Convert the instrument identifiers list to an array:
    InstrumentIdentifier[] instrumentIdentifiers = instrumentIdentifiersList.ToArray();

    return instrumentIdentifiers;
}

Note also that this helper method includes the list conversion to an array.

 

Finally, to save the compressed data file, we call a helper method:

dssClient.SaveCompressedData(rawExtractionResult, gzipDataOutputFile);

Here is the helper method:

public void SaveCompressedData(RawExtractionResult rawExtractionResult, string gzipDataOutputFile)
{
    //We must tell the server we accept compressed data, otherwise it will send uncompressed data:
    extractionsContext.DefaultRequestHeaders.Add("Accept-Encoding", "gzip");
    DssStreamResponse streamResponse = extractionsContext.GetReadStream(rawExtractionResult);
    using (FileStream fileStream = File.Create(gzipDataOutputFile))
        streamResponse.Stream.CopyTo(fileStream);
    extractionsContext.DefaultRequestHeaders.Remove("Accept-Encoding");
    Console.WriteLine("Saved the compressed data file to disk:\n" + gzipDataOutputFile);
}

 

Full code

The full code can be displayed by opening the appropriate solution file in Microsoft Visual Studio.

 

Build and run

Don’t forget to reference the DSS SDK, and to set your user account in Program.cs !

 

Conclusions

This tutorial showed how instrument identifiers can be read from a CSV file, and how to log errors in the process. It also showed how extracted data could be written to a CSV file.

More importantly, it also compared the use of two endpoints; one that delivers JSON formatted data, for easy direct treatment, the other that delivers the data as a compressed CSV for effective download and storage of large data sets.

Now move on to the next tutorial, which continues building on the present tutorial by overcoming the limit on the number of instrument identifiers:

  • Manage a list of more than 75000 instruments, using loops.

 

Tutorial Group: 
.Net SDK Tutorials