Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion GroupsWindows Server 2003Windows 2000Windows NTSmall Business ServerVirtual ServerExchange ServerIISHost Integration ServerISA ServerSMSWSUSMOMWindows Media ServerSecurityCertification
Related Topics
SQL ServerMS WindowsMS OfficePC HardwareMore Topics ...

Windows Server Forum / Exchange Server / Design / January 2006

Tip: Looking for answers? Try searching our database.

Hardware Recommendations

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Ryan Burrus - 10 Jan 2006 21:22 GMT
I am trying to find some information as to the recommended hardware
configuration for Exchange Server 2003.  We only have about 150 users and all
access it using Outlook 2003.  I have read a few things from Microsoft that
says the Exchange data files and the page file should be on separate drives.  
Also that the Exchange data files and the Windows system files should be on
separate drives.  Also that the Exchange transaction logs should be on a
separate drive than the Exchange data files.  This would equate to 4 drives
which doesn't sound right to me.  Does anyone know what the recommended
scenario would be for an organization our size?  Any advice would be much
appreciated.  Thanks!!
Jonathan Norris - 12 Jan 2006 16:21 GMT
It is still best practice to seperate the logs, DB, and OS.  Actually it
would equate to more drives than 4 if you use RAID which is recommended.

Smaller environments sometimes don't always follow best practices since it
does require a larger server and more hardware.  I have seen people use a
single RAID 5 for everything.  However its not considered Best Practice, but
it is still supported by Microsoft.

You will be taking a larger risk with DR because if you loose the Single
RAID container then you loose everything.

If you do decide to go with less disks I highly recommend you do backups to
tape for sure and do it at a daily minimum.

Since I am a consultant I always go with Best Practices and don't recommend
anything less.  

Signature

Jonathan
No Warrenties Implied, Did you do a FULL backup today??????

> I am trying to find some information as to the recommended hardware
> configuration for Exchange Server 2003.  We only have about 150 users and all
[quoted text clipped - 6 lines]
> scenario would be for an organization our size?  Any advice would be much
> appreciated.  Thanks!!
Jonathan Norris - 12 Jan 2006 16:31 GMT
Actually if i were to use two Disks for the OS (Raid 1) and two Disks for
Transaction Logs (Raid 1) and three disks in RAID 5 for the Database .STM and
.EDB Files it would be 7 Disks.  

Also keep in mind by having everything on the same spindles you will have an
I/O bottleneck.

OS/Page File is Read and Write intensive
Transaction Logs are write intensive during normal operation and during
recovery its read intensive.
DB File .stm and .edb are both Read and Write Intensive.

Signature

Jonathan
No Warrenties Implied, Did you do a FULL backup today??????

> I am trying to find some information as to the recommended hardware
> configuration for Exchange Server 2003.  We only have about 150 users and all
[quoted text clipped - 6 lines]
> scenario would be for an organization our size?  Any advice would be much
> appreciated.  Thanks!!
John Fullbright - 12 Jan 2006 19:02 GMT
I wouldn't do that, "three disks in RAID 5 for the Database .STM and
.EDB Files it would be 7 Disks.", that is.

Let's assume a 3:1 read/write ratio for tha deatabases.  We'll also assume
10K RPM SCSI disks which are capable of ~85 20ms IOPS.

Where P is the performance of a single spindle and N' is the number of data
spindles in the set:

RAID 5 Read performance = P*N' = 85*2 = 170
RAID 5 Write performance =P*N'/4 = 85*2/4 = 42.5

And we apply the read/write raitio which is required when read and write
performance are asymmetrical.  With a 3:1 ratio we are 75% read and 25%
write:

Performance = 170*.75 + 42.5*.25 =127.5+10.625 =138.125 IOPS

If you were to buy bigger drives and go with just a mirror:

RAID 1 Read performance = P*N = 170
RAID 1 Write performance = P*N/2 = 85

Again we apply the read/write ratio:

170*.75 + 85*.25 = 127.5 + 21.3125 = 148.8125 IOPS

My recommendation - Three mirrors for a total of 6 spindles.  This is about
8% better overall performance on the database array with 1 fewer spindles.
The write performance is a 100% improvement.

Do the math.

John

> Actually if i were to use two Disks for the OS (Raid 1) and two Disks for
> Transaction Logs (Raid 1) and three disks in RAID 5 for the Database .STM
[quoted text clipped - 25 lines]
>> scenario would be for an organization our size?  Any advice would be much
>> appreciated.  Thanks!!
Jonathan Norris - 12 Jan 2006 20:08 GMT
I never use Raid 5, it was just an example.  Typically I use RAID 10 for this
reason.  Then again I always use SAN.  I can't remember using less at a
client site.

The only reason I would recommend a RAID 5 is for more disk Redundancy.

Yes I am familiar with doing IOP calculations.  Raid 5 with 3 disks would
work fine for both Capacity and Performance with these assumptions

150 Mailboxes
200 MB Mailbox (which is large)
10 K RPM Disk
66% Read Weight
.50 IOP Per User (Typically .25 for the average user)  Microsoft uses 2.0
80 % User Concurrency.

You could use a Raid 1, but plan on having downtime if you loose a drive.

Signature

Jonathan
No Warrenties Implied, Did you do a FULL backup today??????

> I wouldn't do that, "three disks in RAID 5 for the Database .STM and
>  .EDB Files it would be 7 Disks.", that is.
[quoted text clipped - 60 lines]
> >> scenario would be for an organization our size?  Any advice would be much
> >> appreciated.  Thanks!!
Ryan Burrus - 12 Jan 2006 20:12 GMT
We currently have a server with 3 hard drives (RAID 5) and 1GB of RAM.  We
are beginning to notice some delays with people trying to access their
mailboxes (usually when opening large attachtments).

> I never use Raid 5, it was just an example.  Typically I use RAID 10 for this
> reason.  Then again I always use SAN.  I can't remember using less at a
[quoted text clipped - 78 lines]
> > >> scenario would be for an organization our size?  Any advice would be much
> > >> appreciated.  Thanks!!
Jonathan Norris - 12 Jan 2006 21:02 GMT
Do you have seperate Drives for the Transaction Logs and OS?

I would expect READ / Write Delays if you had them on the same
Spindles/drives.

This is why it is not Best practice.
Signature

Jonathan
No Warrenties Implied, Did you do a FULL backup today??????

> We currently have a server with 3 hard drives (RAID 5) and 1GB of RAM.  We
> are beginning to notice some delays with people trying to access their
[quoted text clipped - 82 lines]
> > > >> scenario would be for an organization our size?  Any advice would be much
> > > >> appreciated.  Thanks!!
John Fullbright - 12 Jan 2006 21:08 GMT
Open perfmon and collect the physical disk - sec/transfer counter for your
DB & Log drives.  I bet they're not within the recommendations of the MS
Whitepaper "Optimizing Storage Performance for Exchange 2003".

Slow disk is the number one cause of preceived performance problems in
Exchange.

> We currently have a server with 3 hard drives (RAID 5) and 1GB of RAM.  We
> are beginning to notice some delays with people trying to access their
[quoted text clipped - 98 lines]
>> > >> much
>> > >> appreciated.  Thanks!!
Asher_N - 12 Jan 2006 20:10 GMT
Another one of my pet peeves. Similar to the old, 'the arrow will never
reach the target' problem. While your number are sound, any decent RAID
controller has on-board cache. writes are cached and delayed until read
are satisfied.

So while the theorical problem points to fsaster writes with mirror, if
you use the same drives and same RAID controler to do R1 or R5, you
should get similar results. Also, 150 users is not much. I run 100 users
on a single 5 drive partitioned as 2 logical drives (OS, Stores and
logs). The performance is more than adequate.

> I wouldn't do that, "three disks in RAID 5 for the Database .STM and
>  .EDB Files it would be 7 Disks.", that is.
[quoted text clipped - 60 lines]
>>> recommended scenario would be for an organization our size?  Any
>>> advice would be much appreciated.  Thanks!!
John Fullbright - 12 Jan 2006 21:13 GMT
Cache only goes so far.  Beyond a base amount, adding more cache is a case
of diminishing returns.  If you do rely on cache, it needs to power
protected to prevent corremption in the case of power failure.  The main
advantage of cache in RAID 5 is to remove 1 disk access from the 4 required
for each write.  This effectively changes the formula for write performance
from P*N'/4 to P*N'/3.  Still worse than a mirror.

> Another one of my pet peeves. Similar to the old, 'the arrow will never
> reach the target' problem. While your number are sound, any decent RAID
[quoted text clipped - 71 lines]
>>>> recommended scenario would be for an organization our size?  Any
>>>> advice would be much appreciated.  Thanks!!
Jonathan Norris - 12 Jan 2006 21:20 GMT
I am using a calculator developed by Exchange Rangers and have used it
without any issues (both theoretically and in the real world)  It uses the
same calculations as John Fullbright had illustrated with more considerations.

I ran the information you provided, if you can provide your average mailbox
size, User Concurrency, Disk Size, Disk RPMs, I will rerun it and give you
the projections.

Interesting enough it doesn't even provide recommendations for RAID1 on the
DB spindles.

Agreed Raid 1 does provide more performance, but I would expect to be given
my pink slip if I recommended it as a customer solution for a Database due to
the lack of redundancy and high risk we would assume.

Unless your users have a really high IOP Profile or really huge mailboxes I
wouldn't expect RAID 5 to be a bottleneck (unless you have your OS and
Transaction Logs sitting on the same Disks/spindles).  Which isn't Best
Practice or Recommended by anyone I know (With all due respect).
 
You may also consider running Microsoft Best Practice Analyser to see if it
gives you any recommendations.  Perhaps you have write cache enabled on the
drives?

I would also recommend you run performance monitor to see what the
bottleneck is before you spend money.

Here is a link you may want to check out for Troubleshooting Performance
with Exchange.

http://www.microsoft.com/technet/prodtechnol/exchange/2003/library/e2k3perf.mspx

Signature

Jonathan
No Warrenties Implied, Did you do a FULL backup today??????

> Another one of my pet peeves. Similar to the old, 'the arrow will never
> reach the target' problem. While your number are sound, any decent RAID
[quoted text clipped - 71 lines]
> >>> recommended scenario would be for an organization our size?  Any
> >>> advice would be much appreciated.  Thanks!!
John Fullbright - 12 Jan 2006 21:47 GMT
I don't understand  what you're talking about when you say " but I would
expect to be given my pink slip if I recommended it as a customer solution
for a Database due to
the lack of redundancy and high risk we would assume."

RAID 1 is 100% redundant, it's mirroring..

The Microsoft Whitepaper "Disk Subsystem Performance Analysis for Windows"
gives a set of criteria where it is appropriate to use RAID 5.
http://download.microsoft.com/download/e/b/a/eba1050f-a31d-436b-9281-92cdfeae4b4
5/subsys_perf.doc#_Toc72126973

Quoting from the paper:

"RAID-5 might be the right choice for a write-heavy workload if the
following is true:

·         The workload consists of large requests in comparison to the array's
stripe unit size (that is, they constitute several complete stripes of data
on average), or the controller has a write-back cache, allowing writes to be
delayed and coalesced into full stripes, and

·         The workload is largely sequential (in terms of LBNs), and

·         The array controller can detect and optimize for full stripe
writes, and

·         The number of spindles is reasonable, as described in "Rules of
Thumb" later in this paper, and

·        Cost is a serious issue"

Exchange databases are a random workload that uses a 4K (small requests) IO
size.  A three drive RAID 5 set uses more spindes at higher cost and
provides lower performance than a mirror.  I don't see where any of the
criteria are met for a RAID 5 recommendation.

The same paper discusses the characteristics of RAID 1/0+1/10:

"    ·         Performance:
       For write requests, both parts of a mirror must be updated, reducing
the available write throughput by >50% for 2-way mirrors, >66% for 3-way
mirrors, and so on. On the other hand, read performance improves         as
the number of mirrors in each set increases.
·         Reliability:
For an M-way mirrored set, all M disks must be lost before data is lost. In
the common case of a simple 2-way mirror, MTBF is now increased because it
is the longer of two semi-independent disk lifetimes. In most cases, the
mirror can be restored quickly (for example, by replacing the disk by hand
or using a hot spare), further reducing the window of vulnerability. For
non-trivial mirrored striped arrays, most multi-disk failures are
survivable.

·         Availability:
If connectivity is lost to one disk, or even M-1 disks of every M-mirrored
set in a striped array, work can still proceed, albeit at a different level
of throughput; failure of a mirror spindle decreases read performance and
increases write performance for that set. When disconnected disks become
reachable again, they can be synchronized back to the corresponding active
disks, thereby restoring the original array characteristics."

Both a RAID 5 set with 3 disks and a Mirror can withstand the failure of a
single drive.  I'm not seeing a redundancy problem that would result in a
pink slip here.  I do see a potential perfrormance problem with RAID 5 that
could however result in that pink slip.

>I am using a calculator developed by Exchange Rangers and have used it
> without any issues (both theoretically and in the real world)  It uses the
[quoted text clipped - 111 lines]
>> >>> recommended scenario would be for an organization our size?  Any
>> >>> advice would be much appreciated.  Thanks!!
Jonathan Norris - 13 Jan 2006 17:07 GMT
I got this from
http://www.microsoft.com/technet/prodtechnol/exchange/guides/StoragePerformance/
1c471676-2312-4ffe-adf2-15a9cfd529c4.mspx


The RAID solution you use should be based on the cost and performance
trade-offs that are appropriate for your company. As a result, in many cases,
more than one type of RAID solution may be recommended for a particular data
storage requirement. General recommendations are as follows:

• Use Raid-1+0 for the transaction log file volumes, database volumes, and
SMTP queues

• Use Raid-1 for the transaction log file volumes, SMTP queues, and MTA
queues.

• In general, Raid-5 does not provide the best trade-off between
reliability/availability and performance.

• Raid-0 is never recommended.

The only concern I have with running RAID 1 is if you do loose a disk you
will have to down the server to rebuild it unless the RAID controller
supports hot swap, then again if you loose the disk in the Transaction Logs
or OS your just as dead in the water.

Interesting enough they don't come out and say RAID-1 is suitable for the
DBs in any part of the article.  

Perhaps MS should write another article for the SMB segment.  Since Raid 0+1
isn't a cost effective solution for 150 users its open for debate.

I am not disagreeing with you that RAID 1 provides more performance than
RAID 5.

However running the numbers... RAID 5 will support the number of users and
the IOPS required to support this many users.  But RAID 1 would use less
disks and provide more performance.

So your solution would be better in this case.  I spend so little time in
the SMB segment that I never implement this scenerio.  I stand corrected:)

Here is another article for the person who posted the question,

http://www.microsoft.com/technet/prodtechnol/exchange/guides/E2k3HighAvGuide/c18
04c14-c099-4207-b6b9-de5bda972b76.mspx




Signature

Jonathan
No Warrenties Implied, Did you do a FULL backup today??????

> I don't understand  what you're talking about when you say " but I would
> expect to be given my pink slip if I recommended it as a customer solution
[quoted text clipped - 175 lines]
> >> >>> recommended scenario would be for an organization our size?  Any
> >> >>> advice would be much appreciated.  Thanks!!
Al Mulnick - 19 Jan 2006 02:38 GMT
"Interesting enough they don't come out and say RAID-1 is suitable for the
DBs in any part of the article.

Perhaps MS should write another article for the SMB segment.  Since Raid 0+1
isn't a cost effective solution for 150 users its open for debate.

I am not disagreeing with you that RAID 1 provides more performance than
RAID 5."

Jonathan, I see that you came to the conclusion that RAID 1 was pretty good
in this case.  Since you're on the learning track, can I suggest that you
rethink the RAID concepts as you've come to know them?  Also, rethink the
way that controllers deal with RAID.  Cache is fine but keep in mind that
Exchange is cache-unfriendly.

Also, when you read those documents that folks write, keep in mind they
write them for the general audience.  Although I've personally seen some of
the documents for the SMB segment (and they're pretty good if I say so
myself), don't expect Microsoft to publish papers that say that RAID 1 is
recommended for Exchange.  Will it work? Yep.  Is it as good as RAID 5?  For
some types of operations, yes.  For others, no.  If you have read heavy
applications, you'll get better performance from a RAID 5 than you will from
RAID 1.  Write performance would be less.  Both would have the same type of
fault tolerance.  I throw this out there hopefully to be helpful.  Besides,
you wouldn't have to count on tools from Rangers that way ;)

Johnathan (?), I'm surprised to see in your calculation a number of 2 for
the number of spindles in the data set.  "RAID 5 Read performance = P*N' =
85*2 = 170"  Can you expand on that?  I see two, but it's a raid5 set
meaning you have three possible targets.  Are you saying that only two are
valid for both read and writes? Traditionally, I would have expected 3 for
the write and 2 for the read to account for the parity penalty. Curious. I
realize you're the guy to know, but I'm interested in knowing why that
number is used vs. another.

Cheers,
Al

>I got this from
> http://www.microsoft.com/technet/prodtechnol/exchange/guides/StoragePerformance/
1c471676-2312-4ffe-adf2-15a9cfd529c4.mspx

[quoted text clipped - 260 lines]
>> >> >>> recommended scenario would be for an organization our size?  Any
>> >> >>> advice would be much appreciated.  Thanks!!
John Fullbright - 19 Jan 2006 19:54 GMT
Al Mulnick.  Wow, another blast from the past.  Still hangn' with Jim at
Bubba's Bait & Tackle?

Anyway:

P = Performance of a single spindle in IOPS
N = Number of spindles in the RAID set.
N' = Number of data spindles in the RAID set (N - parity spindles)

RAID 5 performance calculations:

Write performance = P*N'/4
Read performance = P*N'

(depending on the caching architecture implemented by the controller and the
amount of write cache, in some cases write performance can be expressed
P*N'/3)

RAID 1/10/0+1 performance caluclations:

Write performance = P*N/2
Read performance = P*N

RAID 4/RAID DP (as implemented by Network Appliance) performance
calculations

Write performance = P*N'
Read performance = P*N'

For a 3 drive RAID 5 set, N' would be 2.  The set size would be three and
the other spindle is parity.

> "Interesting enough they don't come out and say RAID-1 is suitable for the
> DBs in any part of the article.
[quoted text clipped - 309 lines]
>>> >> >>> recommended scenario would be for an organization our size?  Any
>>> >> >>> advice would be much appreciated.  Thanks!!
Al Mulnick - 22 Jan 2006 00:29 GMT
Ah.  That'd be it.  I think it was a mistake on my part to use a different
caching/controller thought. Must be using the wrong devices :)

Al

> Al Mulnick.  Wow, another blast from the past.  Still hangn' with Jim at
> Bubba's Bait & Tackle?
[quoted text clipped - 352 lines]
>>>> >> >>> recommended scenario would be for an organization our size?  Any
>>>> >> >>> advice would be much appreciated.  Thanks!!
John Fullbright - 22 Jan 2006 07:06 GMT
Some controllers try to coalesce writes, and advertise less of a penalty.
This is where you have to be careful.  For sequential writes, coalescing
writes works really well.  For random workloads, it usually amounts to so
much FUD.  Think about it.  Envision your 100G exchange database as a 1000
page novel.  Coalescing writes is like trying to write an enire sentence
instead of a word at a time.  If you are sequentially writing the next
chapter, writing a sentence at a time works well.  A random workload writes
one word on line seven of page one, and the next on line three of page four
hundred and eightyseven.  The next is, well random; you get the idea.  There
is no real opportunity to coalesce writes so the effectiveness of this
feature for random write workloads is very low.  Unless, that is, you never
overwrite.  If you mark the word on page one as dirty, then update a pointer
in a virtualized structure that represents a sentence to point to the next
available open spot, and you do this for every write, then you essentially
make a random workload look like a sequential workload to the disk
controller.  There is only one storage vendor that does this.

> Ah.  That'd be it.  I think it was a mistake on my part to use a different
> caching/controller thought. Must be using the wrong devices :)
[quoted text clipped - 368 lines]
>>>>> >> >>> Any
>>>>> >> >>> advice would be much appreciated.  Thanks!!
Al Mulnick - 22 Jan 2006 16:43 GMT
Right.  I was thinking of RAID in terms of using available spindles at any
given time.  The more I think about it, RAID 5 would always have the penalty
because it would always have to update the parity as part of the cycle.  But
I was looking at it from the point of view that a controller doesn't have to
only write to the two and then leave the one dedicated to parity.  Nor would
it want to in case that's the one that's gone.  However, I was thinking of
the controllers that keep track of the least busy disk and write to it vs.
thrashing the others.
I've seen some strange configurations ;)

Still, I appreciate the explanation.  Thanks John.

> Some controllers try to coalesce writes, and advertise less of a penalty.
> This is where you have to be careful.  For sequential writes, coalescing
[quoted text clipped - 393 lines]
>>>>>> >> >>> Any
>>>>>> >> >>> advice would be much appreciated.  Thanks!!
John Fullbright - 22 Jan 2006 17:22 GMT
" However, I was thinking of  the controllers that keep track of the least
busy disk and write to it vs. thrashing the others."

Again, this stratgy could work for sequential writes, but won't work for a
random write workload unless you never overwrite.

If you're looking for a disk subsystem that is capable of beating the RAID
write penalty, then the features you want to see are:

1.  Never overwrite, always write to new space.
2.  Coalesce writes across a stripe
3.  Calculate parity once for a stripe in RAM.

You'll see a few vendors doing #2 or #3, but only one vendor does all three.

> Right.  I was thinking of RAID in terms of using available spindles at any
> given time.  The more I think about it, RAID 5 would always have the
[quoted text clipped - 408 lines]
>>>>>>> >> >>> Any
>>>>>>> >> >>> advice would be much appreciated.  Thanks!!
John Fullbright - 22 Jan 2006 23:59 GMT
I'll be in your neck of the woods in early February.  Give me a call or drop
me a line.

John

> Right.  I was thinking of RAID in terms of using available spindles at any
> given time.  The more I think about it, RAID 5 would always have the
[quoted text clipped - 408 lines]
>>>>>>> >> >>> Any
>>>>>>> >> >>> advice would be much appreciated.  Thanks!!
Al Mulnick - 30 Jan 2006 01:54 GMT
I'm guessing you don't check your home email very often? :)

> I'll be in your neck of the woods in early February.  Give me a call or
> drop me a line.
[quoted text clipped - 420 lines]
>>>>>>>> >> >>> Any
>>>>>>>> >> >>> advice would be much appreciated.  Thanks!!
John Fullbright - 30 Jan 2006 03:13 GMT
Hmmm.  I actually check once every week or so.  You can ping me at the
comcursed address if you are having difficulty.

> I'm guessing you don't check your home email very often? :)
>
[quoted text clipped - 427 lines]
>>>>>>>>> >> >>> Any
>>>>>>>>> >> >>> advice would be much appreciated.  Thanks!!
Al Mulnick - 31 Jan 2006 03:12 GMT
I have 3x now.  I'm guessing there's something else.  How about the other
direction? You can send to this one once you remove the extras.

> Hmmm.  I actually check once every week or so.  You can ping me at the
> comcursed address if you are having difficulty.
[quoted text clipped - 437 lines]
>>>>>>>>>> >> >>> size? Any
>>>>>>>>>> >> >>> advice would be much appreciated.  Thanks!!
John Fullbright - 22 Jan 2006 22:04 GMT
"I am using a calculator developed by Exchange Rangers and have used it
without any issues (both theoretically and in the real world)  It uses the
same calculations as John Fullbright had illustrated with more
considerations."

The "Exchange Rangers", all 68 of them collaborating to produce one
calculator.  The mental image of Adrian, Scott, Lee, Ace, Elias, etc. all in
one room is enough to make you go blind.

Wasn't that Greg D.'s *other little band of merry men, besides the
Blackbelts?

Makes you wonder; what came first, the chicken or the egg?

>I am using a calculator developed by Exchange Rangers and have used it
> without any issues (both theoretically and in the real world)  It uses the
[quoted text clipped - 111 lines]
>> >>> recommended scenario would be for an organization our size?  Any
>> >>> advice would be much appreciated.  Thanks!!
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.