One of the products I use is SMSMSE 7.10 (Symantec Mail Security for Microsoft Exchange) and recently I wanted to configure a couple of Edge Role servers with SMSMSE 7.10. After I installed SMSMSE on all the Edge servers my next step was to configure control from one central place with one SMSMSE console – oh boy, I was in for a ride I won’t forget soon.
Before we begin
This will be more of a guide of how to tackle the problems if you ever run into situation like I was in. This will not be typical install this, click here – that part is straight forward, what isn’t straight forward is to get SMSMSE console to work across network which does not have domain.
This is done on Windows Server 2019 with Exchange Server 2019.
A little more about my setup.
Let’s say for this story that I have two Exchange Edge servers – these Edge servers are not in a domain – they are in the workgroup in isolated DMZ network. They have only connection to DC and CAS servers in LAN for DNS and few LDAP ports, so I can use them by name and communicate with them (important for Exchange functionality).
Edge machines have also disabled local Administrator account, I created custom Administrator acoount (InfoAdmin). Since there is no domain, it is imperative that every machine has exact same username, with exact same password and group membership!!
I installed full SMSMSE 7.10 on both Edge roles and wanted to centrally manage them from one machine – I tested with client machine within DMZ and from LAN.
If you wish to connect from LAN to DMZ to manage SMSMSE, make sure port 8081 is let through your firewall but on network/router and in VM.
For this guide I’m going to connect from ExTest1VM (which has installed SMSMSE) to ExTest2 (which also has installed SMSMSE)
If you wish to manage SMSMSE installs from one client that has no Exchange on it, and would act only as a console, you can simply start SMSMSE installer, select Custom install
and install Server management Console only.
Ok, now that we have SMSMSE and our central console in place, we can begin and add our first server (ExTest2) to our console.
After I logged in to the SMSMSE console, I selected Assets from Home menu
After that I selected Global Group – Exchange 2019 and after that Add server(s)… from Tasks menu
I got error, no domain… I really don’t care, since I can find my servers by name, although they are Edge servers in workgroup in DMZ (check out my Exchange Edge and pFSense DMZ guides)
OK
In the “Server name or IP” I’m going to enter “ExTest2” which is the name of the server I wish to join. After that click on >>
I will select SMSMSE 7.10 running on Exchange 2019 | OK
No problems, my server is added to the Selected servers list. OK
Ok, close the Assets menu
Now, on the Home screen select Change… button
Expand Global Group – Exchange 2019 and select ExTest2 | click on Select
Now, remember well above procedure because you are going to do it many times. Removing and then reading server to make sure it works after you changed something.
Ok, I entered my InfoAdmin account (it is main admin account for the VM) and password for it. OK
Ok, first roadblock we got from SMSMSE – The request failed with HTTP status 401: Unauthorized.
Fair enough.
I got through this one fairly quickly. SMSMSE needs IIS to work, so it installed IIS.
We will now go to ExTest2 VM (to the machine we wish to connect via SMSMSE console) and open IIS Manager, inside IIS Manager expand Sites and select Symantec Mail Security for Microsoft Exchange and from the middle screen select Authentication – this is what you will see – Anonymous, ASP.NET and Forms authentication.
Basic, Digest and Windows Authentication are missing.
On ExTest2 start Add Roles and Features from Server Manager and on Server Roles screen under Web Server | Web Server | Security | select Digest, Basic and Windows Authentication | go through the installer – reboot machine after that.
Ok ,now we have new authentication options on ExTest2 VM inside IIS – navigate again to Symantec website in IIS on ExTest2 and select Authentication. Enable Windows authentication and restart IIS. You will have to disable/enable Windows Authentication to provoke login screen again in the SMSMSE console, after you are not successful with the login.
Ok, lets head back to ExTest1 and retry SMSMSE console connection to ExTest2
Again – error. This one is, shall we say – less descriptive. You literally have no clue what is going on. I tried various login options ExTest2\Infoadmin, I shorten my password… Nothing helped.
What is frustrating about this – besides that there is no obvious error message and nothing in Windows/Symantec logs, is the lack of documentation that covers this installation scenario (if there is, I apologize, I wasn’t able to find it)
Ok ,so, let’s go one by one, back to the ExTest2 to test few things.
Let’s first see if the SMSMSE Admins and Viewers group exist on ExTest2 – they do.
Next let’s see if our InfoAdmin user is member of SMSMSE Admins group – it is. Account is also in Administrators group – so all is good.
After hour or two of browsing through the internet and official documentation for SMSMSE on Broadcom site and trying various things, my hunch was that I’m still battling permissions, so I did following
SMSMSE Admin and Viewer group does not have any rights on Symantec folders in the system.
So I added full rights for SMSMSE Admins to the following folders:
C:\Program Files\Symantec (check if the subfolders got same rights you assigned to the top folder)
C:\Program Files\Symantec\CMaF
C:\Program Files\Symantec\SMSMSE
C:\Program Data\Symantec
C:\Program Data\Symantec Shared
(Program Data is hidden folder in the root of your system drive).
I had some issues assigning rights in Program Data folder ( I also have Symantec Endpoint Protection installed) I just clicked on Continue…
Problem is, even after this – I had problems, I would get empty error screen.
And then, out of desperation I went even through txt documents inside SMSMSE installer and found this little gem inside readme file – I never open those inside any installer. I do read the documentation and guides, but readme inside installer dir of an app – never. I guess there is first tiem for everything in life.
Ok, so I opened regedit (as admin) on ExTest2 server and went to following keys
[HKLM]\SOFTWARE\Wow6432Node\Symantec\InstalledApps
[HKLM]\SOFTWARE\Symantec\SMSMSE\7.10\Server
Right click on value – select Permissions
Again, give both SMSMSE Admins and Viewers rights and select Apply | OK. D othis for both registry values.
Reboot the server.
Ok, I was eager to try connecting now from ExTest1 to ExTest2 server.
Maaan, still no luck. Such a frustration.
I also rebooted ExTest1 server, and tried after that to connect again to the ExTest2, I got same blank error, and after I clicked on Cancel, I got something new. Al least a descent error message.
“Access is denied. User is unauthorized or has limited rights on server.
So, we are still battling with permissions. Exhausting.
With this one, Symantec was bit more helpful.
Click on search and enter dcomcnfg on ExTest2 server. Run it as Administrator.
Expand Component Services | Computers | My Computer | DCOM Config (a message will pop-up, I selected Yes) in the middle screen find SMSMSEGUI Class, right click on it and select Properties
Go to security tab.
I set Launch and Activation Permissions | Access Permissions and Configuration Permissions to Customize. Clicked on Edit on each of the three and gave full permissions to the SMSMSE Admins and Viewers.
Repeat process on each of the three edit buttons | full rights for both Admins and Viewers
Apply | OK when you are done, and close everything. Reboot the server for a good measure.
Ok, back to ExTEst1 to test connection to ExTest2
We are finally able to connect!!!!
What an exciting moment. Let’s quickly edit something and Deploy changes just to make sure everything is ok.
Deploy changes
SMSMSE service is not started :( Will this error series ever end!
Ok, back to ExTest2 server to see the status of the SMSMSE service
Service is running on ExTest2
And is running as Local Account which is ok
If I check on home page SMSMSE on ExTest2 server I can see following
Service is started
But, if I check same place on ExTest1 server which is now connected to ExTest2
Status is unknown. “Wonderful”
Logs or error messages didn’t help with this one, also this time there was no hidden readme files anywhere which would help me.
What I tried with
These are some of the things I tried to resolve this “SMSMSE service is not started” error – that didn’t work!
Firewall – I was so desperate I brought it down on both servers – no luck.
More permissions on countless places – didn’t work
After some more reading about SMSMSE, I tried simple thing inside Windows Server 2019 from Powershell (as admin). I tried to access service from ExTest1 server to Extest2
I tried two services wuauserv (Windows Update) and SMSMSE service SMSMSE
Get-Service -Name SMSMSE -ComputerName ExTest2
The results were weird – I could access wuauserv but not SMSMSE.
I went on to download Sysinternals Process Explorer tool and added full rights to the SMSMSE Admins and Viewers to that process – but still no luck.
Windows Server 2019 changed philosophy, so you can no longer easily access services on another machines. So I disabled that on ExTest2 quickly by adding following to the registy (run in cmd as admin)
reg add HKLM\SYSTEM\CurrentControlSet\Control /v RemoteAccessExemption /t REG_DWORD /d 1 /f
I also tried to add SMSMSE exception to RemoteAccessCheckExemptionList key on ExTest2 server in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurePipeServers\SCM
After that I rebooted ExTest2 VM and I was even able at some point to access SMSMSE service remotely from ExTest1 server on ExTest2 – But anyways – SMSMSE console would just not give up. SMSMSE status was unknown when connected remotely.
What worked for the SMSMSE service is not started in the end?
After many exploration and reading, and few more hours lost I got to the bottom of it, and solution was simple
As a desperate measure I enabled Administrator account on ExTest2 VM. I added built in Administrator also in SMSMSE Admins group. Built-in Administrator has same rights as my InfoAdmin. Exact same (as I can see)
After that I re-added ExTest2 to SMSMSE console on ExTest1 and provoked (via IIS Windows Authentication disable/enable) again login screen to login with another user. This time I used Administrator user. If you get Error 401 while entering Administrative username – check if Windows Authentication in IIS is enabled under Administrator account. If you don’t get authentication screen like the one below – remove server from SMSMSE, disable Windows Authentication on the server you wish to add, then re-add the server so you can provoke login screen again.
I was able to login and SMSMSE Service was showing as started!!! What a joyful day.
I was finally able to deploy change I meant from Extest1 to Extest2.
What a day.
Conclusion
Situation that I described above was to say the least – frustrating. Deploying SMSMSE on Exchange Edge role is supported scenario. What should be better covered is this scenario – where Edge is in DMZ, without domain and extra machines around it. Since Edge role is more exposed to the internet, you want to harden it as much as possible. Leaving default Administrator account on is not my first choice on exposed machines.
Other solution is to just manage SMSMSE from each machine individually and not to have central point for management and configuration deployment.