Architecture - Server Spikes & Application Hangs

Asked By santhosh kumar on 12-Oct-14 02:59 PM
I have my application hosted in Amazon Cloud. We were facing an issue for last 30 days intermediately.

We have 2 DB servers on 2 different instances for 2 different customers
We have 2 app server to host wcf services for load balancing. The app servers are common for both the customers.
We have 2 Back office servers for administration person which rarely communicates with the wcf service/app server.
It directly communicates with the DB's
We have 2 Web service servers.
We have 2 front end web servers for UI whcih communicates to app servers through amazon internal load balancer.

Note:-
There is a load balancer before front end  which schedules request in round robin manner.
The services can communicate using nettcpbinding with web server and across any servers.

There is a sudden sharp spike across Web Server 1 , Back office 1, Database 1 & Database 2. Then after 2 minutes there was a sharp rise at Web server 2 and Back office 2.

But the spike is not consistent it came down instantly but thereafter one of application went down.
However, all the other applications were working as expected.

Can you suggest what could have been the problem if any body experienced similar pattern.