Develop an ASP.NET API site together with an Angular JS frontend

Recently I’ve started to play around with Angular JS and ASP.NET core. I’ve created an ASP.NET web API project and (basically) copied the Angular frontend to wwwroot.

All Angular stuff is running by using an index.html file containing the app and all pages are displayed by using an MVC like scheme (aka http://www.example.com/customer/list). API calls to the backend services are encapsulated within the path /api (e.g. http://www.example.com/api/customer).

After creating a web API project with Visual Studio, it’s not possible to drive static files which are copied to wwwroot. Therefore two more lines are necessary to drive files from within an API app. Please add the following lines before the app.UseMvc() call in Startup.cs:

…
// switch to index.html on any request not starting with "api/"
// (C# controller) nor being an existing static file
app.Use(async (context, next) =>
{
  await next();
  var path = context.Request.Path.Value;
  var fileInfo = env.WebRootFileProvider.GetFileInfo(path);
  if (!path.StartsWith("/api", StringComparison.InvariantCulture)
      && !fileInfo.Exists)
  {
    context.Request.Path = "/index.html";
    await next();
  }
});
app.UseDefaultFiles();
app.UseStaticFiles();
app.UseMvc();
…

The first statement (app.Use(...); ) will handle all not known requests to the backend and route them to index.html (e.g. the angular router will handle the request).

Any request that is not routed via api  and that does not map to an existing file under wwwroot  will be forwarded to index.html. Routing errors (e.g. invalid pages, …) are forwarded to /404 .

app.UseDefaultFiles() will check for default files if a root path or directory is given. In the current scenario this will work only for the root directory which is sufficient.

app.UseStaticFiles() is responsible to serve all files located under

Example code can be found at https://github.com/steven-r/angular-cli-webapp-api.

Protecting your customer … from unnecessary requests

Today I stumbled over a twitter post:

This one is referring to the “product deathcycle”. This picture shows one part of the problem. Another part are requirements which cause the customer to be unhappy about you (the vendor). Every business analyst or requirements engineer knows about those customer demands.

So… Why are some requirements worse than others or why do they even harm the relationship to you.

IMHO requirements that are creating “un-happiness” are those which solve the problem that was on top of the customers mind while the key account (or business analyst) was visiting the customer. I’ve seen so many occurrences where a key account came back from the customer site with dozens of new ideas where most of them are “urgent”, “out of discussion”, “essential for our users” or even “important for our relationship”. Anyhow, the last one is somewhat an exception because that is usually a request to get something for free ;).

What is happening here that most of those urgent requirements result out of a single incident or escalation currently on the desk of the customer. If you solve this problem (probably included in a release several month later) it is out of the customer’s mind and usually therefore not urgent anymore.

Try to evaluate (together with the customer or through experience) if the customer’s requirement is a lasting one (no one time shot, really required and not part of a single escalation). In any other case try to see if the requirement can be

  • cancelled in the first place (i.e. convince the customer that he doesn’t need this requirement),
  • replaced by something the customer really demands (is lasting, improves efficiency, …),
  • cancelled during the offer process or
  • (as a last resort) deny to provide an offer for a specific requirement

The latter two ones are usually the ones with the most negative feedback from the customer. They should be avoided by using one of the first two options.

I cannot remember how often a customer came around with an important report the management (or somebody else) demands. After further evaluation the usual cancellation reasons where:

  1. One time demand (results in one day in Excel or two weeks in software),
  2. Exists already in slightly different form and is sufficient or
  3. Can be easily created in Excel with existing reports or data

Having your first hybrid development project with your new customer

… or “Rome wasn’t build in one day”1

Recently I’ve been invited to speak about hybrid development processes in the context of business analysis in Aachen.

One of the points discussed was how to implement a hybrid development process for a new customer especially if

  • the customer does not have any experience with agile or hybrid process models or
  • the customer’s procurement process does not allow agile or hybrid processes for external projects.

The simple conclusion and recommendation after a longer discussion was to start with something simple that the customer (and you) can handle.

Start slowly

We’ve seen good results with an approach that uses as many features and parts of the hybrid development process the customer can handle at a time; especially if you develop something with a customer who doesn’t have any deep knowledge of agile methodology start slowly.

In the first project we usually stick with the regular waterfall approach (requirements, contract, develop, deliver, accept) and use our agile process internally for development.

While going on with new projects and enhancements implement hybrid features by discussing changes open minded while developing the software. Start with some kind of fuzzy-ness for requirements and pass over to alternatives or a complete agile approach.

Do not rely on a stable customer relationship

Another point: Things change, this also happens while working in an active customer relationship.

Processes, resources and their minds change, and so the hybrid process needs to be adapted to the changes.

Do not think that if the hybrid process is implemented you can use it for a long time without change. If the relation to the customer changes in any of the above cases you may need to go back to start and re-implement the process again.

Footnotes

Using datatables.net with ASP.NET webforms – A generic approach

datatables.net is a flexible HTML table component used in many applications. Recently (in a large ASP.NET webforms legacy application) we’ve had the need to switch from a commercial product towards a product which is easier to maintain and which does not contain 5000 features where we use only 10 out of them (and some of the others are creating problems).

Using datatables.net is very easy and requires only limited knowledge on JavaScript in case you are providing table data together with the data to be displayed. You can find tons of examples on the datatables.net home page.

In case you are going to display data which is

  • hard to generate on the fly (volume, complex queries) or
  • is changing during paging or other table operations

you need to load data from the server even after the page has been requested. The term used in datables.net is “server side processing”.

By using a .NET handler class and a generic interface that encapsulates the data provider we’ve generated a solution which can be easily adapted to create a generic data table provider implementation in ASP.NET web pages.

I’ve created a short demo on github that shows the general approach and might help as a base to use an own implementation.

Hybrid Software Development Process – Writing Flexible Requirements

18-02-_2016_21-58-19During the last posts (here and here) I’ve discussed the overall problem and the hybrid software development process in general. This post will detail the requirements part of the hybrid model.

How requirements are covered and business analysis is performed is discussed in several books, articles and reports somewhere else (please have a look at this search or IIBA for some information).

The basic problem is that several parts of a requirement are either

  • not yet finished (i.e. the user department cannot give a final commitment on all parts of the requirement) or
  • the customer has not yet decided on several options to fulfill a specific requirement

As this information is necessary to create a binding offer while using a regular process/project model, it may be left over in a “fuzzy” state with some restrictions that allow us to create an offer based on the given information.

Any “fuzzy” information known at this point in time is usually more stable than any information from the requirement’s author when the author is forced to give a final statement on a requirement while.

So, instead of insisting that the requirement needs to be finalized, those points are marked as “open” or describe several (similar) “option”s.

Open Requirements

Open requirements are requirements which allow a concrete estimation for the offer but leave enough space to define the details on this particular requirement later during the development phase.

An example of an open requirement would be a report where the data to be shown on the report is defined but the format/layout is still open.

Another example of an open requirement would be a set of rules to define several levels of contribution levels. It is clear how many are there but which account accumulates to which contribution level can be defined later on during development.

It is very important to define the rules and margins on every open requirement to enable the vendor to create a estimation on this requirement that can be used as part of a fixed price offer.

Defining Options

Options define similar functionalities (from the user’s perspective) which allow the customer to decide later on which option to take. The options may have similar efforts.

An example for options is to decide on CSV or Excel (XLSX) format for an import later during the project. In case if efforts are not similar, the customer needs to be aware that as of the fixed price project only one price tag will be communicated and the price doesn’t change in case the customer decides on the cheaper one.

How to manage open requirements and options during later phases of the project

It is very important that the customer is aware of the fact that there has to be a decision to be taken for each and every open requirement and option during the development process. It has to be agreed (latest) during the contractual phase if and when the decision needs to be done. If the decision is not there at the point where it is needed project delays or additional costs may occur.

Effective Meetings

After participating a (phone) meeting that was not worth spending the money for the IP traffic, I’ve sent a mail to all participants referring to the following post: Tips for Running Effective Meetings. Some of the folks replied and asked if the infographic is available as text as well… Here we are:

  • Email an agenda 24 hours in advance
  • Arrive 5 minutes early
  • Come prepared
  • Bring paper and a pen
  • Start and end on time
  • Stay on topic
  • No interrupting
  • No smartphones
  • Be brief and concise
  • Share all relevant data
  • No side conversations or comments
  • Disagree without being disagreeable
  • Everyone participates
  • Silence = Agreement
  • Challenge ideas rather then people
  • Follow-up email within 24 hours

Looking at all those points I need to work on some of those points :)

The original infographic:
intuit-quickbase-tips-for-running-effective-meetings-infographic

The hybrid process – general design

The first post in the series on hybrid software development talked about general problems many software development companies have during their customer relationship. Contracts are very often based on a linear process as shown below:
Hybrid Contract flow

In addition, user departments responsible for the requirements and the software may have the need to change the requirements even after they have been finalized or contractually agreed due to several reasons. Some of them are:

  • A change in another process requires a change of the requirement
  • Some requirements have been forgotten
  • A change due to e.g. legal reasons need to be included

Based on the number of required changes projects need to be shifted, deployment dates need to be moved or projects already started need to be cancelled.

In the last decade agile development tried to get rid of some of the challenges mentioned above. Based on the “Agile Manifest” a lot of development processes have been designed that try to solve the problems mentioned above. As a result, requirements are created at a rougher level at the beginning and later on detailed during development at the time when they are really developed. In recent years several of these process models have been designed, namely Scrum, FDD and others.

In recent years it has been seen that even those approaches have raised questions and concerns. Some of them are:

  • They do not follow enterprise processes (as enterprises are very often stick to linear processes where there’s a contract required with a fixed price offer based on a detailed specification)
  • Very rough requirements at the beginning might cause legal intervention in case of later delays not non-wanted results
  • Customers (and requirement engineers) need to be more involved during the complete development process.
  • Processes like scrum or other agile methods tend to deliver software in more frequent deployments (to reduce time to market). A software delivery with intermediate (and therefore changing) features need a high amount of change management. This becomes heavy if you use a complex software on a high user base.
  • Team building is a more complex process as agile teams with a given velocity tend to react negative to changes in stuffing.

We have designed a hybrid process which tries to overcome (most of) the concerns described above and create a model combining the best out of the two worlds (classic and agile).

The hybrid process will change each of the process steps above in a way to achieve a possibility to create flexibility during development for a fixed price project:

  • Requirements: Allow fuzzy requirements that remain estimatable
  • Contract: Enhance the contract process to recognize fuzzyness and allow reserves
  • Develop: Agile (e.g. Scrum) and allow refinements
  • Deployment: (Usually) only one deployment towards the customer
  • Acceptance: Acceptance based on requirements and refinements

This process has been established now for medium to large projects for at least three years with increasing success.

Further posts will discuss details on every process step and will provide more in-depth guidance.

Hybrid software development processes – Introduction

This blog post belongs to a couple of (yet to be written) posts about hybrid software development processes. please follow the blog to get updated with the latest blog posts.

At the time we’ve started our company some 10 years ago agile methodologies had not reached the development mainstream at software development companies or IT departments. At that time we decided to go the “classic” way by using an iterative waterfall (see here pg. 329f – fig. 4) approach.

Learning about agile development and new team members made us moving towards Scrum in 2009. The agile approach gave us the flexibility we’ve looked for and we liked the innovative chances of this new process.
We ran a couple of projects with Scrum, some went well, some did not. We learned a lot about success factors, advantages and disadvantages.

Analyzing the project results we had seen there were a lot of cases where we had a clash between the classic “request” > “offer” > “order” > “produce” > “accept” way of handling projects towards the customer and the agile process. We learned that it is difficult to apply the Scrum process all the way to the customer. Customers often prefer the classic approach, especially when it comes to timing, outlines and the commercial aspects of a project. On the other hand late changes in a development project are often requested.

With the waterfall process late changes are difficult to cover, with Scrum they are easily implemented but do have negative impact on time and budget. This negatively impacts customer satisfaction and causes uncovered expenses and delays.

We thus tried to merge the advantages of the waterfall and the scrum process and created our own hybrid process using parts of both methodologies:

  • Towards the customer we stick on a classic linear process (iterative waterfall) to have a known and established way of communication towards our (mainly) enterprise customers and
  • internally use Scrum as the development method to allow flexibility in respect of work package building and
    continuous evolving of requirements into stories and tasks

This process even allows the customer to incorporate late changes as long as

  1. the requested requirement to be changed has not been developed already and
  2. the change fits into budget, time and quality constraints.

There’re some other constraints but basically these are the rules.

If one of the above rules are not matched, even then the change might be accepted but in this case we have to come back to the customer and explain him that this change will have side effects (budget, time, …) and the outlines (schedule, price, etc.) of the project need to be changed However many of the smaller changes that occur in almost every project can be covered without any commercial change.

The next blog post will discuss the overall process followed by blog posts related to specific parts / topics related to the process itself.

Reading streamed data from Excel or CSV

Recently I stumbled over the task to read either CSV or Excel files and transfer data in an independent format to a server.

The existing software part implemented to similar approaches to

  1. Read the file  
  2. Transfer the file into an intermediate format (a DataTable)
  3. Transfer the file up to the server

The server took the file, read all data, translate number according to a given number format sent by the client, …

Reviewing the code resulted into a couple of drawbacks: redundant code, performance, use of language depending information on the server side, etc.

As a result I looked for an approach to

  1. combine CSV and excel load within one flow and  
  2. have data translated on the client side into a format which can be read by the server without knowledge of the client user language (or the date format used for loading files)

Based on a code example for IDataReader (http://blogs.msdn.com/b/anthonybloesch/archive/2013/01/23/bulk-loading-data-with-idatareader-and-sqlbulkcopy.aspx) I had the idea to create a library which streams data independent of the input format.

The result is now on GitHub. A library which results into a simple interface to read data:

Continue reading Reading streamed data from Excel or CSV