Today I'd like to go over a simple Rust CLI application I made.  It is set up with just one simple command, however it is organized to scale to many different commands. Please feel free to use it in your own work.  If you have any suggestions, please send me an email - mitchgollub@gmail.com.

The technologies I'm using today are below

• Rust
• clap
• tokio

Rust CLI Example App Overview

The project is organized into a few sections

.
├── Cargo.lock
├── Cargo.toml
└── src
   ├── cli.yaml
   ├── commands
   │   ├── js_to_ts.rs
   │   └── mod.rs
   └── main.rs

Under src we have the cli.yaml file.  This holds the definition of our application and it's commands.  At link below, you'll see a command to change Javascript files to a Typescript file simply by changing the extension. There is an option to specify a directory to search recursively.

cli-example/cli.yaml at main · mitchgollub/cli-example

Contribute to mitchgollub/cli-example development by creating an account on GitHub.

GitHubmitchgollub

main.rs holds the application orchestration code.  You'll see some code that translates string input to a Command.  The program first runs some setup code to configure logging and error reporting.  After that, it parses the CLI input and runs the appropriate command.

cli-example/main.rs at main · mitchgollub/cli-example

Contribute to mitchgollub/cli-example development by creating an account on GitHub.

GitHubmitchgollub

The final file of note is the js_to_ts.rs file under the commands folder.  This is all the code that runs when the command is selected.  The file here shows a simple command that searches a folder recursively and changes any files with a .js extension to a .ts extension.  It uses tokio tasks to leverage some system concurrency.

cli-example/js_to_ts.rs at main · mitchgollub/cli-example

Contribute to mitchgollub/cli-example development by creating an account on GitHub.

GitHubmitchgollub

Checkout the full source and give it a ⭐ on Github if you've found it useful. Thanks for reading! 📖

I heard a lot of great things about Rust.  The promise of a C-like language with guaranteed memory safety sounds too good to pass up.  It's super fast, and seemed to be a useful language to learn given you can write code for embedded systems, command line applications, and front-end and back-end web applications.

What is Rust?

On their website, Rust states their intentions are to create a language that empowers everyone to build reliable and efficient software.  It's major features include memory and thread safety, which is a common area for bugs to show up in code.  Rust is created and supported by Mozilla and can be found here: https://www.rust-lang.org/.  Another helpful resource is their thorough tutorial they affectionately call "The Book": https://doc.rust-lang.org/stable/book/title-page.html.

Rust on AWS Lambda

All that being said, why not set up an AWS Lambda to run my Rust code! Rust is not a supported runtime for AWS Lambda, so this will also be good experience working with custom runtimes (https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html).

Luckily, AWS Labs has done a lot of the leg work getting Rust to run on Lambda.  Check out the links below for more information.  Be wary, as I chose to use the master branch of lambda instead of the current release (v0.3.0) in my code sample. The latest release is missing an important recent update at the time of writing this for using shared data between lambda runs. However, you can use the v0.3.0 release without this update using the AWS Labs Github examples.

• AWS Blog post on the Rust Runtime for Lambda - https://aws.amazon.com/blogs/opensource/rust-runtime-for-aws-lambda/
• Documentation on running Rust using SAM - https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/building-custom-runtimes.html
• GitHub issue describing the shared data updates - https://github.com/awslabs/aws-lambda-rust-runtime/issues/310

Let's get to the code!

Let's take a quick walk-through of the code linked at the bottom of this article.  The file structure is below

.
├── Cargo.lock
├── Cargo.toml
├── events // folder to hold Lambda JSON events
│   ├── google.json
│   └── turnip-exchange.json
├── README.md
├── run.sh // script to run Lambda code locally 
├── src
│   ├── client
│   │   └── mod.rs // surf client code
│   └── main.rs // AWS Lambda Rust runtime code
└── template.yaml // SAM template for Lambda 

main.rs has the code the sets up the AWS Lambda Rust runtime.  There are more examples and explanations on this code at the links in the previous section.  

mod client; // module for surf client wrapper

use client::RustLambdaHttpClient;
use lambda::{handler_fn, Context}; // AWS Lambda runtime for Rust
use serde_derive::{Deserialize, Serialize};

type Error = Box<dyn std::error::Error + Send + Sync + 'static>;

/// The custom incoming Lambda event structure
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
struct CustomEvent {
    host: String,
    path: String,
    http_verb: String,
    #[serde(default)]
    post_data: String,
}

/// The custom outgoing Lambda event structure
#[derive(Serialize)]
struct CustomOutput {
    message: String,
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    // shared client to be used across lambda invocations
    let client = RustLambdaHttpClient::new();
    let client_ref = &client;
    
    // entry point for the lambda event
    lambda::run(handler_fn(
        move |event: CustomEvent, _ctx: Context| async move {
            dbg!(&event);
            let url = format!("{host}{path}", host = event.host, path = event.path);
            let body = client_ref
                .clone()
                .send(event.http_verb, url, event.post_data)
                .await?;

            Ok::<CustomOutput, Error>(CustomOutput {
                message: format!("Body: {}", body),
            })
        },
    ))
    .await?;

    Ok(())
}

Making an HTTPS request

A working sample of Rust running on Lambda is available on my Github. It parses an incoming event to make a simple HTTP request using surf.  You can run the lambda locally in a docker container using the run.sh script available in the repository.  Deployment to your AWS environment can be done through the SAM CLI tool from AWS.  

I've separated the client code to a client module.  The code for that is below

use crate::Error;
use surf::http::Method;
use surf::Client;
use surf::Url;

/// Wrapper used to share the surf client struct
#[derive(Clone)]
pub struct RustLambdaHttpClient {
    client: Client,
}

impl RustLambdaHttpClient {
    pub fn new() -> Self {
        RustLambdaHttpClient {
            client: surf::Client::new(),
        }
    }

    pub async fn send(
        self,
        http_verb: String,
        url: String,
        post_data: String,
    ) -> Result<String, Error> {
        let mut res = self.client
            .send(
                surf::RequestBuilder::new(parse_http_method(&http_verb), Url::parse(&url)?)
                    .body(post_data)
                    .header(
                        "User-Agent",
                        "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:79.0) Gecko/20100101 Firefox/79.0",
                    )
                    .build(),
            )
            .await?;
        let body = res.body_string().await?;
        dbg!(&body);
        Ok(body)
    }
}

/// Parses HTTP method codes from the Lambda Event
fn parse_http_method(input_method: &str) -> Method {
    match input_method {
        "GET" => Method::Get,
        "POST" => Method::Post,
        _ => panic!("No matching HTTP method for {}", input_method),
    }
}

Source Code

My source code is below!  Feel free to use it as a template for your own needs.  Hopefully it helps.  Since I'm new to Rust, any feedback is appreciated.

mitchgollub/rust-lambda-https

Contribute to mitchgollub/rust-lambda-https development by creating an account on GitHub.

GitHubmitchgollub

Other Links

• https://docs.rs/surf/2.2.0/surf/
• https://aws.amazon.com/blogs/opensource/rust-runtime-for-aws-lambda/
• https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html

Today we're going to look at a simple set up to get AWS Lambda Functions running in VS Code.  You'll be able to run your code locally in a Docker container with the Lambda runtime and debug your function through the VS Code debugging tool.

My development environment is an Ubuntu Linux laptop with Node.js code running in the Lambda.  These tools are open source and cross platform, so you should be able to install these tools on a variety of systems.  However, if you are running another AWS Lambda supported language, such as Python, you might need to change some configurations to match your codebase.

Prerequisites

VS CODE

Visual Studio Code is a cross platform IDE developed by Microsoft.  You can find download links and instructions here: https://code.visualstudio.com/Download

AWS SAM CLI

The AWS Serverless Application Model (SAM) CLI tool is a program that allows you to scaffold, run, and deploy serverless applications.  You can find download links and instructions here: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html.  The SAM CLI will require Docker as well.  Documentation is available on that guide to get it installed.

Creating your Serverless Function with SAM

The SAM CLI allows you to create a new lambda project with defined Templates.  The tool makes it easy to set this up with this command.

sam init

Follow through the prompts to create your application.  I chose a Nodejs12.x application.

Package.json

I had to edit the package.json file in the SAM template to make this work.  Below are the npm start and npm run debug commands I created to run the SAM environment with NPM.

The most important sections are lines 9 and 10.  These will actually launch the SAM CLI tooling instead of Node.js.  The debug script will specifically be used to hook into the VS Code Debugger tools.  

Launch.json

This is a crucial configuration for setting up these two tools to work together.  The launch.json is a file that VS Code uses when setting up the debugger.  I have my code posted below, and I'll walk through the important parts.

The type is node, so if you are using a different programming language, you'll need to use an appropriate configuration in VS Code.  On line 9 is cwd or current working directory.  You'll need this to work with the application structure generated by the SAM CLI.

For debugging, the lines to add are 18 and 19 for localRoot and remoteRoot.  This will allow VS Code to map your local JS files to the JS files running in the Lambda container.  

Once you have the files set up, give it a try!  Set a breakpoint in app.js and launch the VS Code Debugger.  You should see the execution pause on your breakpoint and have the ability to inspect variables.

Hope this was helpful!  If it was or you want to make some suggestions, feel free to send me an email at mitch.gollub@gmail.com.  

However, my 10" Chromebook is!  I decided to see if it was possible to use this little guy as a lightweight coding machine.  After doing some online research, I found these tools to be immensely helpful in getting a Development Environment set up for node.

Notes on Crouton

One popular option I found for creating a development environment on a Chromebook is to use a Linux distribution.  While I'd most likely prefer using Ubuntu or another type of Linux OS for development, my goal was to get an environment on the smallest system I own.  That being said, I had two problems with the Crouton approach: security and performance.  Crouton requires running ChromeOS in Developer Mode.  While it's relatively harmless to do this, I do enjoy the sandbox security that comes baked into ChromeOS.  In addition to security concerns, I'd most likely experience pretty poor performance on my Chromebook's rudimentary hardware.  With that, let's move on to the tools I chose to use.

Termux

Termux is a Linux terminal emulator for Android devices.  If your chromebook can run Android apps, then this should work for you.  It runs pretty well on my ASUS C100 with a Rockchip processor and 2GB of RAM.

Text Editors

It's worth installing a text editor as well.  Termux comes with Vim and has a package for Nano, so those can be used if you're interested.  For those not so comfortable on the command line, Caret is a good option.  However there are countless other text editors for Chromebooks you can try.

Okay, now we're ready for some development!  Let's get Termux configured and try to launch a simple react app!

Install Node

Termux has it's own package manager.  You can search that list by typing pkg search <keyword> or view the whole list with pkg list-all.  You can also get this info by visiting their page.

At the time of writing, Termux offers two NodeJS installations: nodejs and nodejs-lts.  I'm going to play it safe and install the LTS pkg, but either will work just fine.  Go ahead and install with pkg install nodejs-lts

Creating a React App

Installing the NodeJS runtime will also give you the NPM package manager!  Unfortunately there are some issues around running create-react-app in Termux.  I was able to reference this stackoverflow article to get it to work.  Follow the steps below to create a project called chromebook-react.

npm cache clean --force
mkdir chromebook-react
touch chromebook-react/.gitignore
npx create-react-app chromebook-react

On a slower machine like mine, this may take a while.  In the meantime, feel free to pick up an energizing hot beverage ☕

Running the Development Server

Great!  Now you can set up the development server for React the same way you would normally: npm start.

There is a quirk about running the server in the Termux emulator.  Even though the displayed address will be something like http://localhost:3000, your Chrome browser will not be able to access the server via localhost.  You will need to get the IP of the Termux terminal.  

You can do this by typing ipconfig in the window and seeing Termux's IP address.  It's been a consistent value of 100.115.92.2 at the time of writing this.  So, once the server is up, you should be able to access the React page by typing http://100.115.92.2:3000/ in your browser!

And that's it!  You should be able to edit some files in the React project and see them change on save in your browser.  This environment won't work for heavier workloads, but it's really nice to have the flexibility to do development work on a cheap 11" chromebook.  

I hope you found this useful!  Please feel free to share your thoughts with me at mitch.gollub@gmail.com

High Order Components (HOCs) in React are functions that, given a component, enhance it with some extra behaviors, returning a new component.

Usage

A quick example would look like the sample below:

An example for using a HOC would be if you want the same attribute to appear on multiple components such as className.  You could write a HOC such as the one below:

Note the "with" prefix on the name of the HOC.  It is common practice to prefix HOCs that provide some information to the components they enhance using this pattern.

Recompose Library

The Recompose library provides many useful HOCs as small utilities that we can use to wrap our components, moving away from some of their logic.

Check out some of these useful HOCs:

flattenProp()

Sometimes the objects we work with are structurally complex.  You might want to flatten the structure of incoming objects to make them easier to work with in your component.

renameProp()

The renameProp() HOC allows for you to rename a passed prop for use in the component.

You can also combine HOCs to constructed an enhanced component for you application

Performance

Remember to use HOCs with caution.  While they make code much easier to read, the trade off for using HOCs is in performance.  Wrapping a component into a higher order one adds a new render function, a new life cycle method call, and memory allocation.

Recently I went through the React Tic-Tac-Toe tutorial that's available right on the React website.  While it's not a complete, enterprise-grade crash course in using React, I feel it teaches some important concepts in React and refactoring components.  You even get a neat game out of completing it too!

Components

React is known for being component-based.  This helps immensely when trying to increase code reuse and follow the DRY Principle.  The component aspect is great for companies that would want to share UI objects across projects.  Many teams could share a React Component library to save time by not having to rebuild a UI Element.  This concept also helps unify the look and feel of your apps.

State Management

The game tutorial focuses mainly on component state management.  It starts out with three components: the square, the board, and the game.  You'll begin by putting logic in the square for handling the onClick() event.  As you add features, it becomes apparent that the state of the square will need to be known by its parent components. The tutorial then guides you through moving the state from the square component.  This is referred to as Lifting State, since you are "lifting" the function and values from the child component to be defined by the parent.  This allowed for more control when trying to determine whether a player has won the game, to store a history of moves, and to move the game back in time to those previous states.

The Result

The result of the tutorial is probably the best part.  I pushed my code to now to host and share the project.  You can find the link to the game here: https://my-react-tutorial.mitchgollub.now.sh/. Thanks for reading!  

I'm currently working with a lot of Extract, Transform, and Load (ETL) type problems at my job.  Someone has data they need to get from point A to point B and maybe have some processing done in between.  That being said, the input data in this equation can take many forms.  Sometimes those forms can take on a state that crashes an application and possibly halts the ETL processing altogether.  In this series, I'd like to talk about some defensive programming best practices on how to prepare your application for non-ideal situations that very often take place in the real world of ETL processing.

String types are pretty straightforward on ingestion.  Most files and sources that we read as input or write as output can be cast into Strings directly. Most issues I've seen come in processing Strings, which I'll talk more about in an upcoming section.

Conversion

Recently I was trying to debug a process written in C#. It takes data from an Excel file and transforms it into a fixed-width text file with transformed data.  The data coming in can be of a couple different types, but must be output as a string to be written to the text file for a third party vendor to digest.  The application was running into a "gotcha" that comes up in .NET around transforming data into a String.  

The Object is the ultimate base class used by classes in .NET.  This means we could be returning ANY class from the getValueFromField() function and try to cast it into a string.  If a type other than System.String is retrieved, you might get an error such as this one: System.InvalidCastException: Unable to cast object of type 'System.Double' to type 'System.String'.

.NET has a .ToString() method that is much safer to use than a type cast.  The method is implemented on many classes, ensuring that you're much more likely to guarantee that you can produce a String type.  Below was what we introduced to allow for processing to continue.  Note the null-conditional operator.  You can read more on that here: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/member-access-operators#null-conditional-operators--and-

It looks like the data type for that incoming field had changed to being interpreted as a Double at some point.  However, we still want to write that field to the output text file.  Making this change allowed the file to be processed without error.  While the program runs, ideally we should leverage the strong typing in C# and use a new method like getDoubleFromField() to make sure we aren't passing our logic a generic Object class, which was the root cause of the issue.  The reason we did not, is because there is no processing that really requires the System.Double type for this field.  Simply going from a string field on the Excel sheet to a String in the output file was the most defensive option.  

There's so much more to talk about on this topic, so I'll be updating it as I write more.  Stay tuned for my upcoming sections on parsing String data. If you liked what you read, let me know!  Feel free to reach out to me at mitch.gollub@gmail.com.  Happy coding!

Today, I'd like to show you a quick CI/CD project I put together using Blazor.  This app is made from the sample template for a client-side Blazor project.  I built and deployed it using Azure Pipelines.  It's hosted in Azure using Blob Storage and the Static Websites feature.

What's Blazor?

Blazor is a .NET framework that leverages WebAssembly to run .NET code in the browser.  You can build client-side applications with the logic built in C#.  Blazor allows you to set up even more complicated architectures using a client-server model and SignalR (https://docs.microsoft.com/en-us/aspnet/core/blazor/hosting-models?view=aspnetcore-3.0#server-side).  For this project I just made a simple single-page application (SPA) using the default Blazor template.  You can find more about how to build the template for yourself here: https://docs.microsoft.com/en-us/aspnet/core/blazor/get-started?view=aspnetcore-3.0&tabs=visual-studio.  Please note I created this project with the .NET Core 3.0 preview version 3.0.100-preview3-010431.  The project template structure changes in later versions of the SDK so this may not work if you use a different version.

What's Azure Blob Storage and Static Websites?

I chose Azure Blob Storage because I wanted to try the Static Websites feature.  A Blazor SPA would be a perfect fit to cheaply host in Azure Blob Storage and still have the power of C# and .NET.  

I set up the storage account in Azure and enabled Static Websites.  This created a $web folder that I could target using my Azure Pipeline build.

Creating the Azure Pipeline build

Azure Pipelines allow developers to build and deploy their code all from the same platform.  They offer hosting for code repositories and even test plans for QA, but I'll just be using the Build and Deploy services for this project.  

I was able to pull my GitHub code for the Blazor project and start building the project the same way I would on the command line with the .NET CLI.  The Pipeline Build will even trigger once the master branch is updated.

The key to building the Blazor project was using the .NET Core SDK Installer task.  I was able to tell it exactly what version of the .NET Core SDK I needed for my Blazor template to build correctly.  From there, it was as simple as invoking dotnet build, dotnet publish and storing the build artifacts.  You can even put a step to run your tests during your build process, but I have that disabled until I get some tests running.  

The Pipeline Release is super simple as well.   We just want to Extract the Files from the Artifact, then copy the files to our Storage Account using the AzureBlob File Copy Task.  This Release Definition is configured to run after a successful build.

Once that's set up, you can push to master and watch the magic happen.  After the Release deployed successfully, you should see some files in your $web folder in Azure Blob Storage.  

You can go to the address where your Static Website is mapped and should see your Blazor app run if everything went smoothly.

I have some links below to the site and source code.  Check them out!  The webpage even has a Microsoft Cognitive Services Computer Vision integration (https://azure.microsoft.com/en-ca/services/cognitive-services/computer-vision/).  So if you input your API key and an image URL on the Computer Vision page, you'll get back some results.  

Source: https://github.com/mitchgollub/PhillyCodeCamp2019.1-Blazor

Site: https://blazorstaticapp.z20.web.core.windows.net

Thanks for reading!

I recently began working with AppVeyor as a deployment platform for my .NET deployments.  It has proved to be a great tool for implementing build and deploy automation for .NET Framework and .NET Core applications since it treats .NET as a first-class citizen.  Most configurations with AppVeyor are almost automatic, however I was missing automated variable substitution by deploy environment that I became so comfortable with when using other continuous integration tools like Octopus Deploy.  

Why do I need Variable Substitution?  

It allows for making environment specific Connection Strings, API credentials, and other configurations separate from your application code.  This software architecture idea comes from the third section of the tweleve factor app methodology, which you can read more about here: https://12factor.net/.  You'll find this feature in a lot of modern web frameworks, not just in ASP.NET Core.

Let's go over how Octopus Deploy handles appsettings.json variable substitution. For a JSON based file, Octopus Deploy has an out-of-the-box feature to apply configuration values per environment.  It can replace root level values, hierarchical properties, and even array values using a special syntax.  More documentation on how you can implement it in Octopus Deploy and how it works here: https://octopus.com/docs/deployment-process/configuration-features/json-configuration-variables-feature.  

AppVeyor does not support this directly in their product for JSON files.  They do support variable substitution using a parameters.xml file to target a web.config file in an ASP.NET deployment (https://www.appveyor.com/docs/deployment/web-deploy/#parametersxml).  However, it seems Microsoft is moving towards JSON based configurations with .NET Core.  

The Script

The way I got around this was by leveraging a part of the AppVeyor deployment pipeline.  AppVeyor uses a Powershell script named "deploy.ps1" to drive and automate any deployment tasks on the target machine.  I've written a Powershell script to replace appsettings.json configuration values to match your environment.  (If you're having trouble with the script, please see the update at the end of this article.)

The script utilizes the user-defined environment variables that AppVeyor writes to the target machine for deployments.  I use a prefix ($env:APPLICATION_PREFIX) for grabbing an Array of those variables, then loop through them.  Each iteration of the loop searches the target appsettings.json file (or any other JSON based configuration file you feed it) for matching properties.  If a property matches the name of an environment variable (with the environment prefix stripped out), then the value of the environment variable will be written to the output file.

Now what this doesn't support that you can find in Octopus Deploy is the array variable substitution.  Currently this handles root level and hierarchical property values, but I'd like to add array support some time in the future.  I'd also like to expand this to use recursion so it can support more than a property depth of one.

Feel free to use this script for your own purposes.  I had trouble finding one like it, so hopefully you find this suitable for what you need to get started. If you find a way to improve it, I'd love to hear about it!  Send an email to mitch.gollub@gmail.com. Thanks for reading!

UPDATE 1/31/21:

If you've been having trouble with the above script, @eivindivine was able to make some adjustments and get it working.  He shared his example here:

Check out the link to his gist for more information on what changed!

Hey all!  

Got an assignment recently for an interview to a company handling investment and share selling data.  They actually gave me a well thought out problem that they have to solve all the time prior to the interview.  The requirements were to take a CSV file of Market Transactions and generate reports based on the data.

The Assignment

Now, I only had 2 days to work on the submission, and I was working full-time during those days.  I also probably haven't updated the project since then, so just a little warning there.  Below are the requirements.  I probably only hit about 60% of the bullets.

Summary: Write code to consume data in a CSV file, and generate reports.

You may use any language, libraries or tools you want, as long as you will be
able to demonstrate your solution in person. Code that you can email to the
interviewers ahead of time usually works best, but other means of
demonstration, such as a laptop with the necessary tools loaded, would be fine
as well.

Input Specification: CSV file. First line is a header. Columns are:
    TXN_DATE - date transaction took place
    TXN_TYPE - type of transaction (BUY/SELL)
    TXN_SHARES - number of shares affected by transaction
    TXN_PRICE - price per share
    FUND - name of fund in which shares are transacted
    INVESTOR - name of the owner of the shares being transacted
    SALES_REP - name of the sales rep advising the investor

Output Specification:

    1. Provide a Sales Summary:
        For each Sales Rep, generate Year to Date, Month to Date, Quarter to
        Date, and Inception to Date summary of cash amounts sold across all
        funds.

    2. Provide an Assets Under Management Summary:
        For each Sales Rep, generate a summary of the net amount held by
        investors across all funds.

    3. Break Report:
        Assuming the information in the data provided is complete and accurate,
        generate a report that shows any errors (negative cash balances,
        negative share balance) by investor.

    4. Investor Profit:
        For each Investor and Fund, return net profit or loss on investment.

Be prepared to discuss:

    Testing strategies
    Difficulties you had and how you solved them

Suggestion:

    Your code should be correct first, have good style and comments second, and
    then be clever/fast as a distant third.

The Code

I chose to write the project with a .NET Core back-end to consume the CSV Data and an Angular front-end to display the reports.  The .NET Core server code uses the CSV Helper library to consume and generate strong-typed entities from the CSV file.  I do this here in the code: https://github.com/mitchgollub/DotNetCore.Angular.SalesCSVReader/blob/05c78f9f3dd7b130e9ad566dc387b4ff7bf762ec/Repositories/TransactionRepository.cs#L24-L34

CSV Helper is written and maintained by Josh Close and is taking PR's on GitHub.  More info on CSV Helper here: https://joshclose.github.io/CsvHelper/.

Writing the reports in Angular helped me easily grab aggregated data from the back-end server and display it in tables on different pages.  I could define a ViewModel for each report with all of the completed columns and have the server send over the rows.   That happens here: https://github.com/mitchgollub/DotNetCore.Angular.SalesCSVReader/blob/05c78f9f3dd7b130e9ad566dc387b4ff7bf762ec/ClientApp/src/app/sales-rep-summary/sales-rep-summary.component.ts#L11-L23

Things I would have done differently

So, given that the CSV was rather short, I had not run into any performance issues running the solution as it was.  However, the architecture would easily crawl to a halt when aggregating a large dataset.  There are two changes I could make to allow the application to scale: Reduce iterations over the dataset and minimize the amount of times the code reads the CSV file.

One example of the iterations I did here shows four aggregates that pass over the data multiple times.  

I can reduce the number of passes through the data by leveraging a LINQ Aggregate.  The Aggregate calculation will be heavy in that it would calculate the total monetary value from the transaction using the CalculateTransaction function, then check each transaction for a date matching date range to be added to the totals for Year-to-Date, Month-to-Date, etc. However, with a significantly large dataset, the performance benefit of passing the data only once will outweigh the extra computation.  More on the Aggregate function here: https://docs.microsoft.com/en-us/dotnet/api/system.linq.enumerable.aggregate?view=netcore-2.1

Another point I would improve would be the code reuse on the Angular app.  Each component is a copy and paste job from the original .NET Core/Angular template project from Microsoft.  Given more time, I'd break out the HttpClient calls to a ReportDataService that could be injected to each component.  Then there could be a function on the ReportDataService that calls a wrapper function around the reused HttpClient.get() commands.  I could even merge the .NET Core endpoints into one Controller, return all the report aggregate data in one ViewModel, and just make one HttpClient call.  Merging the endpoints would reduce code duplication in the .NET Core app as well.

The final major update that could be implemented would be some graceful failing in the event that the CSV structure changes.  As the app stands at the time of writing this, it is semi-resilient to CSV changes.  If new columns are added, they will simply not be picked up by the code.  However, if the CSV file moves, the existing column names change, or existing columns are removed, the server will throw a 500 error when trying to return a report.  

Implementing this resilience might be a little more involved.  I'd imagine it'd be good to store a backup CSV file from the last successful run of the application.  That way if the file has an error during data retrieval, we can fall back on the backup file.  This will hide the issue, so proper logging and notifications should be put in place to alert the party involved with the maintenance of the CSV file can make adjustments.  If the business need requires showing that the file is ill-formatted, we should bubble that message up to the UI (in a friendly way) so the user is aware and can handle the situation accordingly.

In short, working with CSV files in .NET Core isn't so bad.  CSVHelper made reading the file incredibly easy.  I'd still prefer a database to store the data, but for some custom integrations, CSV's are a very real part of people's workflows.  The project itself was impossible to complete in a short amount of time, but I'm happy with what I was able to make.  One day maybe I'll go back and it round out with the items I listed that I would change.  

Hey gang, this my first post on the blog.  Just getting my feet wet with the platform.  I'm hoping to be able to get into a regular rhythm of writing on programming, music, and card games.  

I wrote some code today, actually!  Just a small project that takes a structured XML file inside of a .NET Core MVC project and displays it in a table.  It was my first introduction to DataTables in JQuery, which made it super easy to put in a bunch of different constraints that'd be pretty complicated otherwise.  

Check it Out!:

https://github.com/mitchgollub/DotNetCore.MVC.ProductXmlReader