Thats on the small side.... RAM is 64GB minimum for that amount of users and you need flashstorage for System drives and email storage sice searches would take longer on spinning drives.

Calculate the storage size based on the current user average.

Sounds like you are looking to go on bare metal ?

Have you ever used ProxMox ?

Others can chime in on this with their thoughts, but you could consider to virtualize the server on ProxMox. This way you can use proxmox connectivity and potentially use several options, start out basic and expand as needed. We do not have anywhere near that many users, but we do use ProxMox and with the server virtualized, I can adjust the ram, adjust the drive space, repoint the drives to a different location, expand them when necessary, set up time based snapshots, etc.  We have drives in the ProxMox machine, and We also have a few TrueNAS Servers where those are our main datastore. I can point the client data over to teh TrueNAS machines data pool, and set their own snapshot schedule there.  Also, with ProxMox, you can set up several machines as a cluster and set up failover and High Availability so if a machine goes down, it can relaunch on another machine. And using refurbs, that is exactly the hardware we have in place. 3 refurb ProxMox servers in a cluster with SmarterMail running, and TrueNAS as a data store. Just last night I shut down the VM Smartermail machine and reallocated more ram to it, expanded the drive storage and then rebooted and was back up online in less than 3 minutes. You can manage the storage size on TrueNAS independantly, so if later you need to add more drives to it, you can do that, make adjsutments on a different TrueNAS machine, and then migrate over to it.  Even using external storage like a SAN if you wnat.

We run SM virtualised on Debian under ProxMox, brings lots of handy features including PBS integration with nominal performance overhead. We run SSD only but with the tiering you could potentially keep new content on SSD and older content of the spinning rust.

We run everything on VmWare Esxi Enterprise Plus and Flash Storage.

Runs smooth as silk.

I have recently put in a server that was on a tight budget, however this was for 100 users, not your 3000, but might be useful as a baseline.

I went with a Dell T360, 128Gb ram, 8 x 8Tb (SATA) as R10. I simply could not afford SSDs as they want capacity. Server 2025 as Hyper-V host and SM as Linux on VM. I have assigned 8 cores, 48Gb ram and 3Tb VHDX to start with. In terms of performance as I type this, the cores run at around 20%, 12Gb (dynamic) ram

Disc access is barely registering. All on MAPI plus 30 on EAS. average of 4000 emails per day.

RspamD runs 4 cores

I found SM to not be as aggressive on disk access as I was expecting.

In a perfect world we all get the biggest and fastest of everything and price is no object, right?

The SAS RAID using HDDs rather than solid state is fine and offers excellent performance, particularly in RAID10. Sure, SSD is faster and uses less energy but the cost is prohibitive getting into the multi-terabytes in a RAID array, as you mentioned. We have used HP ProLiant exclusively for ages so it's what we know but Dell is just fine, too. Getting corporate off-lease (aka used) rack mount servers is a great idea, and what we do. You can fully load out a great 2U, two socket, redundant power, 8-24 disk capacity server for not much money.

You need more system memory for sure. If you have the budget do 128gb.

My advice is to virtualize your implementation. We went that route nearly 15 years ago and I would never contemplate a "bare metal" server running the mail server again. I don't intend to sound preachy about it, but virtualization makes everything easier. Adding storage to the instance, adding memory, adding CPU, backup, restore, server migration, restarts and maintenance, snapshots prior to changes, adding network adapters... It's absolutely a dream. Abstracting the server from the hardware just makes sense.

In our farm, we have multiple hypervisors interconnected with 10gb adapters, we can move SmarterMail from one host to the other seamlessly with no downtime. Whenever we do a SM upgrade, take a VM snapshot first, perform upgrade, check for function, and delete snapshot. It has saved my bacon a number of times. Rollback takes 3 mouse clicks and a minute or two, if necessary. If I need to add memory? Shut it down, assign more RAM, start it back up, and SmarterMail is back up in barely a minute. I can add storage or expand storage on fly with zero downtime. This is certainly also an advantage if you can't afford SSD or more storage now. If the server has empty chassis slots, just keep them empty until you can afford faster/better storage in the future. Then, just do a live storage migration from array A to array B. Zero downtime and easy future expansion... I'm partial to MS Hyper-V but Proxmox has made quite a name for itself in short order, particularly after VMware's changes in licensing terms. Virtualization is a no-brainer in my book. Off my soapbox now...

So, get the server(s) with the components now that won't require you to shut down and open up the chassis to upgrade in the future. Disk arrays can be added with no downtime on a hypervisor that supports raid adapter management software.

That's my nickel's worth of advice. Good luck, and let us know what and how you do!

Matt

**edit. sorry for the typos. I banged this out in a hurry**

Nathan, Have you worked with ProxMox ? I can share several decent video about it.

You would want a 10Gig network between machines, and the 10G cards are really inexpensive now. Shoot, I have seen 40Gig cards out now.  We buy a fair amount of stuff used from this guy. He tests with TrueNAS and I believe ProxMox too. We have bought NIC cards, raid adapters, pieces parts...  https://www.ebay.com/str/theartofserver  

You could get a SAS Controller and later on expand as much drive capacity as you want externally.

My 10 cents is that IOPS matter on a busy server.

And you dont get that on HDD's.... no matter if they are in a Raid10.

We run everything on VmWare Enterprise Plus and Flash Arrays with 100Gig NIC's on storage and switch uplinks.

We can execute things significantly faster than our competitors here in Europe and we are running SAP better than every other SAP hoster out there.

Thats the story behind a backbone thats working as it should.

It takes 6-7 seconds to boot a server to the point where the login screen appears.

We can reboot a fileserver so quickly that a busy factory with 112 employees havent even noticed it was gone.

And that matters in the hosting world and that matters when you host email services for your clients.

IOPS are definitely critical but it also depends what the client is paying so it is 'horses for courses'.