Interest in implementing ESB

Dec 14, 2007 at 7:22 PM
Hi Minas,
We have a customer ni the US that is in the process of acquiring/developing an ESB tool. We are very interested in ESB.Net, but are kind of stucked in building a proof of concept app (like a Hello World service) with ESB. Waht help can you give us in that direction?

Another topic: the customer needs to take that decision quickly, what are the chances of stablishing a Skipe/MSN conference with you or one of your collaborators to vanish the customer doubts on the features, ESB support, training, etc?
Coordinator
Dec 17, 2007 at 12:15 PM
Edited Dec 17, 2007 at 1:07 PM
Hi,
If you install the download, there are sample services included that work. You can modify them if you like. There are varying levels of samples included:
1)Services using Basic Message Envelopes & no documents
2)Services with Messages using 1-2 documents in the envelope, some including attachments
3)And there are a few services using Windows Workflow for Service Orchestration & the WF Rules Engine etc.
4)There are also Convention over Configuration type services that are attribute driven. Again, these are in the included samples.
In my installation, the samples are under:
C:\ESB\Dev\Source\ESB\LobSystems\

and the configuration over convention ones at:
C:\ESB\Dev\Source\ESB\LobSystems\ns

There's also an online version of ESB.NET running at:
http://60.241.173.62/esb/source/management/default.aspx

You can use the Simple Web Based ESB Management Console to make simple config changes here. More complex changes require editing the XML Config files directly on the file system.
You can also test pushing sample requests through, viewing responses, logging, re-submitting etc...

TIP: When submitting requests, be sure to change to the WSE3 or HTTP Transport as the WCF transport currently has a large delay in setting up the channel (fixed in WCF 3.5).


Also, see
http://esb.net.au/OpenStandards/tabid/138/Default.aspx
for a general background in ESB's.
Often, many of the items covered in the presentation are not done or not done correctly.
The ESB will then either fail to deliver any real value, or fail to take off at all.

Dec 17, 2007 at 1:52 PM
Thanks Minas for your answer.
And are you aware that in the management console there are pages that are not working? Some of them just return de 404 error (file not found) or if they fail, the just show the previous page (so you really don't know if they failed or not, nor why...)
Coordinator
Dec 17, 2007 at 8:07 PM


SheikCod wrote:
Thanks Minas for your answer.
And are you aware that in the management console there are pages that are not working? Some of them just return de 404 error (file not found) or if they fail, the just show the previous page (so you really don't know if they failed or not, nor why...)


Hi Sheik,
Thanks. Any specific ones?
Please note that the pages with the red circle are currently Not Implemented.
Also, the pages under the "Services" node are merely an example of where you can add your own Service specific configuration if you like. This would typically be reserved more for cross-cutting services etc. For example, Authorization, Caching, Duplicate Detection, A Generic Idempotent Service handler etc.
Dec 17, 2007 at 8:25 PM
Thanks Minas, I think most of the cases correspond to what you said.
Dec 17, 2007 at 8:34 PM
About the MSN Meeting, our client is decideing who attends the meeting and when. But to gain some time, I'm sending you the list of desired features defined by our customer, so you can check if them apply to ESB. Net, and if you can give a brief answer the them it'd be most appreciated. The list includes the "titles" of the features, plus a brief description of what the customer understand by each one of them. Also, what's the level of coverarage of the features (from Built-In to Not Included, including Requires al lot/some/small amount of effort)

2.1 Handles Orchestration
Should be able to take a business process from the beginning of its life to the end of its life.
An Orchestration can be as simple as a message coming in from a client (front-end), being directed to a service, capturing the service response, and responding to the client. It may also entail having human intervention required to finish a business process (Faxing/phone call/filing, etc) - the solution must handle this type of orchestration as well.
Some of the features needed in the orchestration requirement would be:
2.1.1 Tracking of processes
Should be able to find out what stage a process is at in it's orchestration
2.1.2 SLA
Should be able to specify Service Level Agreements that must be adhered to during a process.
2.1.3 Replace transactions
Should be able to replace a transaction (Compensation transaction, typical of a "rollback" in a long running transaction)
2.1.4 Restart (at point of failure)
Should be able to restart an Orchestration from the point of failure and not necessary to restart it from the beginning.
2.1.5 Business Processes - Long running/Human Interaction
Should be able to handle Non-automated orchestration processes that may take months to complete
2.1.6 Visible to internal users or Support Staff
Should be able to show internal users the state of a Business Process (Where it's at, what may be wrong, status, etc)/
2.1.7 Management/Monitoring
System must be able to me managed and monitored easily. Alerts should go to those that need them with valid and useful information.

2.2 Sync and Aysnc processes/messaging
Should be able to handle both

2.3 Translations
Should be able to take a message of one type and translate it into another (Flat file,XML, CSV, etc)

2.4 Scale Out
Rather than scaling Up (Bigger/Faster Hardware), the solution should be able to be load balanced or mirrored in some fashion that would allow you to speed up the entire process by adding, not upgrading.

2.5 Open Standards
Should use open standards such as XML, WCF, WS-* and not proprietary formats

2.6 Training/Support Availability
Should be able to take classes/training on the product or have resources available to the company to facilitate issue handling.
The solutions that is chosen should have a "life" associated with it that coincides with the business. An open source project that has long been "dead" would not be a good choice.


3 Other Nonfunctional Requirements
3.1 Performance Requirements
There should be no extraneous steps or processes that are run in order to move through the architecture to get the end result, which is the data/information that is requested.

3.2 Security Requirements
Security at this level will be handled by certificates, WS-*, Active Directory and possible SQL

3.3 Software Quality Attributes
(weas not defined by the customer)


Coordinator
Dec 18, 2007 at 11:30 AM
Edited Dec 18, 2007 at 8:48 PM

{quote}
SheikCod wrote:
About the MSN Meeting, our client is decideing who attends the meeting and when. But to gain some time, I'm sending you the list of desired features defined by our customer, so you can check if them apply to ESB. Net, and if you can give a brief answer the them it'd be most appreciated. The list includes the "titles" of the features, plus a brief description of what the customer understand by each one of them. Also, what's the level of coverarage of the features (from Built-In to Not Included, including Requires al lot/some/small amount of effort)

2.1 Handles Orchestration
Should be able to take a business process from the beginning of its life to the end of its life.
An Orchestration can be as simple as a message coming in from a client (front-end), being directed to a service, capturing the service response, and responding to the client. It may also entail having human intervention required to finish a business process (Faxing/phone call/filing, etc) - the solution must handle this type of orchestration as well.
Some of the features needed in the orchestration requirement would be:

*Apart from rudimentary configuration based orchestration - mostly used for filters etc. - ESB.NET leverages Windows Workflow for Orchestration/Workflow.
There are custom Activities for ESB.NET that ship, new ones in the pipeline, and you can create your own as well as modify/extend the base ESB.NET ones to do pretty much anything you like.

The default usage of WF within ESB.NET is as follows:
- Sequential workflows used for quick orchestrations ("Business Activity Services" in the Keystroke ESB Services MetaModel). These do not maintain state by default.
- State based workflows used for long running processes ("Business Processes" in the Keystroke ESB Services MetaModel). These maintain state.


2.1.1 Tracking of processes
Should be able to find out what stage a process is at in it's orchestration

*For State based workflows, this is a feature of Windows Workflow & hence attainable through the WF API's. Additionally, ESB.NET will track the current state and a few key attributes in a table for quick/easy query based reference etc.
*For Sequential workflows, ESB.NET does not currently store any additional information to WF, so you are currently limited to the base WF functionality (which is actually quite rich).
You could leverage the tracking service for Sequential Workflows to manage the current step in an orchestration. This would require you use some sort of storage though (typically the WF Tracking Db used by the state based workflows). By default, the sequential workflows do not maintain state. Introducing state here can be done selectively, on selected nodes, to selected Tracking DB's. Note that this will affect the horizontal scaling story.

2.1.2 SLA
Should be able to specify Service Level Agreements that must be adhered to during a process.

*The Service Definition specifies the priority of a particular Business Activity Service. CPU & Message priority can be manipulated to match the service definition. This is currently not done out of the box. Rather, you (or there will be in future releases) build & configure a Handler/Filter that runs and sets the thread priority etc. according to the Service Definition value.
There is also timing information in the context that can be obtained and used to raise events (if say, the time taken is greater than a certain value etc.), again by filters. Unfortunately, you have to write the filters (samples available upon request), however, this allows you to do whatever you want.

At the business process level, between the WF API's and the ESB.NET "in progress workflows" table, there will be enough information and enough events to be able to raise alarms of a particular process exceeding a pre-defined time to complete and transition between states. You can also build specifics into a particular workflow if you really needed to.


2.1.3 Replace transactions
Should be able to replace a transaction (Compensation transaction, typical of a "rollback" in a long running transaction)

*This is possible but the "Compensating Transaction" must be Orchestrated into the main orchestration, or run separately as a standalone compensating transaction.

2.1.4 Restart (at point of failure)
Should be able to restart an Orchestration from the point of failure and not necessary to restart it from the beginning.

  • This is not currently available out of the box. It is on the cards in future releases though. It only applies to Sequential Workflows ( and also used to handle state transitions in State Based Workflows).
It will be on a service by service basis, as there will be an overhead associated with this (needs to be stored in tracking database etc.)

2.1.5 Business Processes - Long running/Human Interaction
Should be able to handle Non-automated orchestration processes that may take months to complete

*This is done via State Based Workflows

2.1.6 Visible to internal users or Support Staff
Should be able to show internal users the state of a Business Process (Where it's at, what may be wrong, status, etc)/

*This is currently done via third party WF Viewers. In future releases, this will be built into the ESB.NET Management Console.

2.1.7 Management/Monitoring
System must be able to me managed and monitored easily. Alerts should go to those that need them with valid and useful information.

*The ESB.NET Management Console is used for this & for manual resubmission of failed requests (from the Logs).
Alerts can be configured as filters. Filters can be configured to run for a particular service or for all services.

2.2 Sync and Aysnc processes/messaging
Should be able to handle both

*Can be done at the client (via ESB Envelope request message parameter) and server (using the Pipeline Configuration) side. This is all configuration driven.
.NET based clients can also choose to use the client side async type processing functionality available in .NET.

2.3 Translations
Should be able to take a message of one type and translate it into another (Flat file,XML, CSV, etc)

*Only supports XSLT out of the box.
Code or third party parsers are required for non XML formats.

2.4 Scale Out
Rather than scaling Up (Bigger/Faster Hardware), the solution should be able to be load balanced or mirrored in some fashion that would allow you to speed up the entire process by adding, not upgrading.

*Supported via Network Load Balancing etc.
There are many options here. State based services obviously suffer from scale-up options only.
You could leverage 3rd party distributed memory mechanisms to address this

2.5 Open Standards
Should use open standards such as XML, WCF, WS-* and not proprietary formats

*Is purely internet standards based at the interface level. Internally, it uses many of the latest MS technologies such as WCF & WF.

2.6 Training/Support Availability
Should be able to take classes/training on the product or have resources available to the company to facilitate issue handling.
The solutions that is chosen should have a "life" associated with it that coincides with the business. An open source project that has long been "dead" would not be a good choice.

*The Keystroke ESB architecture was developed in 1999/2000. It has had several implementations spanning C+/VB/COM/MTS/COM, Java/SeeBeyond eGate, .NET1.0/1.1, BizTalk, webMethods .NET 2.0, and the current .NET2.0/.NET 3.0 flavour that is the current version of ESB.NET.

The Keystroke Architecture has not changed much since 1999. The basic principles are the same. This is because the basic business level requirements in terms of enterprise wide services have not really changed. Mostly technology & capability has changed.



3 Other Nonfunctional Requirements
3.1 Performance Requirements
There should be no extraneous steps or processes that are run in order to move through the architecture to get the end result, which is the data/information that is requested.

  • Being lightweight, having multi-instancing support, advocating federation and supporting both stateless and stateful services based upon individual requirements, means that scalability can be sensibly controlled.
For example, you can setup farms of servers for stateless services.
Similarly, you can setup a high availability database server & app level servers for your stateful Business Processes.
ESB.NET does not really restrict you here. Rather, being an advocate of federation means that you should end up with a large number of independent Business Units & related services, all working with a similar capability. You can be free to implement part of your ESB using other technologies. All you need to do is to support a similar set of ESB capabilities in whatever technology you decide to use in a particular instance (which may be a relatively big technical undertaking, but you know it will work, and will fit in with your existing ESB infrastructure - they're usually the hard bits. Anyone can build sutff... Building the Right stuff is the current Industry ESB problem).

3.2 Security Requirements
Security at this level will be handled by certificates, WS-*, Active Directory and possible SQL

*ESB.NET uses WCF, WSE3 & HTTP as the out of the box transports, all running atop IIS. Therefore, all these features are available to you.
AD is used as part of Windows Auth. & SQL currently used for logging & Workflow (although you can change this to use whatever you like).

ESB.NET uses WS-* at the transport level & implemented by WCF & WSE3.

3.3 Software Quality Attributes
(weas not defined by the customer)

*This is a broad area, but:
-the use of a proven pattern & implementation helps you get a solution out the door
-ESB.NET uses the MS App.Blocks as well as third party libraries & much of the MS stack for reliability & scalabitily (eg. IIS, MSMQ)
-Services are built and deployed independent of each other. You can run multiple instances in parallel. Automated testing...

{quot
Dec 18, 2007 at 1:01 PM
Thanks Minas for your answer!
Any comments about bullets 2.6 (apart from the fees you already told me) and 3.1 and 3.2?
And about 3.3 (Software quality), though not defined by the customer, feel free to add anything you consider relevant.

Thanks again.
Coordinator
Dec 19, 2007 at 9:12 AM
Sheik,
I've updated the above post...
Dec 20, 2007 at 6:58 PM
Minas, please if you see this post, connect right away to MSN. Thanks (sorry for using this forum to send you this message)