Issue
Enterprise Server ES crash
Product Line
EcoStruxure Building Operation
Environment
- Enterprise Server ES
- Enterprise Central EC
Cause
The ES/EC runs on a Windows platform and there are circumstances that can lead to a single crash or multiple crashes leaving the ES/EC not running
Background - The ES service is usually running. The service can be seen below under the Windows services and via "Building Operation Software Administrator"
The ES v4.0 has crashed and will not restart.
The "Building Operation x.x Connect Agent" collects any core dumps and packages them, so they can be quickly analyzed. It also checks if an ES is registered and online, then uploads the core dump files for automatic analysis. If the site is not registered or is offline, then these files are stored locally at
%ProgramData%\Schneider Electric [EcoStruxure|StruxureWare]\Building Operation [n.n]\Enterprise [Server|Central]\db\Crash information
(default for v4.0 "C:\ProgramData\Schneider Electric EcoStruxure\Building Operation 4.0\Enterprise Server\db\Crash information")
The "Building Operation x.x Connect Agent" is a Windows Service and installed with the ES, the service should be left to run continually. If it is not running during an ES crash, the core dump may still get created, but does not get packaged up in a way that can be easily analyzed.
Each packed crash dump file may look like "ES_ES_MYCOMPUTER_20221104T121314Z.zip" and likely be between 40MB and 200MB in size.
The individual core dump will look like "server_20230314_132853__r4.0.3.176.dmp", but is very time consuming to analyze, without the other associated files
Resolution
Following a crash,the EC/ES will usually restart and operate correctly, however in some cases the EBO database or install file structure may be damaged and the EC/ES may fail to restart correctly
EC/ES has crashed and is now running correctly
Send the following information to Support
- All core dumps from the "\db\Crash information" folder
- All data from the "\db\logs" folder
- Ideally a full backup of the ES (with history), always zip before sending to Support
If the "Crash information" folder is empty, then check the "Building Operation x.x Connect Agent" is running.
It is possible the core dumps have already been uploaded and analyzed automatically
EC/ES has crashed, tried to restart but just stops again. (as seen in the first screen shot, "The server has made too many unsuccessful attempts to start..." is seen
It is important to gather all the data as stated above, but also collected the EC/ES and AS backups from "\db_backup"
It can be useful to get full copy of the files in "/db"
Ideally the EC/ES is not critical to the operation of a site, but it can still cause a big problem.
Checking the Trace and System logs may give a clue to the problem, e.g. a lack of disk space, but often there are only a few options
- Collect the files above, then wait for a reply from support. Please state if the EC/ES is stopped and a resolution is being sort.
- Reinstall the EC/ES at the same version and choose repair current version, or upgrade to a newer CP Fix version or maintenance version.
- Uninstall the EC/ES, then re-install the same version with an empty database, then restore the last good backup to the ES.
If option 2 or 3 is used, it is still important to log a case with support, so the crash can be investigated.
The backup can be checked for errors using the XBK Analyzer.
