SPECapc SW 2007

It's out:

http://www.spec.org/gpc/downloadindex.html

 

And here are the rules. Anybody care to backup to SP0 to run it?

# Solidworks 2007

1. The benchmark must be run using Solidworks2007 service pack 0.
2. The application window size must not be changed from its initial
size.
3. The submission must contain a result.txt file generated by
running the benchmark. Its contents are extracted from the file
apcresultsN.txt, generated by the benchmark, where N is the number of
the benchmark test run.
4. The submission will be derived from the best composite generated
by the default 5 benchmark runs, as controlled and reported by the
benchmark GUI.
5. The appearance of the the application's Quick Tips / Dynamic
Help box, or other pop-ups that do not dismiss themselves, will cause
the benchmark run to be invalid. The benchmark FAQ has information on
how to prevent this.
6. The directory structure of the submission must be as follows:
.../company-name/system_1/sw2007/result.txt
.../company-name/system_2/sw2007/result.txt
etc...
7. The submission file must be named company_apc_sw2007_vN.zip
where company is the member company or organization name in lower case
and vN is the file version (e.g. ibm_apc_sw2007_v0.zip.) The initial
submission is v0. Resubmitted files must have the version number
incremented.

 

I believe a requirement for sp0.0 was stated in the instructions for
previous versions as well. I'm fairly certain it wasn't enforced in the
software then, and I hope it isn't now. I'm certainly not rolling back for
it, and a lot of people don't have sp0.0 install media.

 

snip..........

I have SP0.0 on my Dell M70, and I'm downloading now....at 8kbps!!!! Only 3 hours to go
Their servers must be saturated.

Art W.

 

It is a requirement in order to submit results to SPEC. If you use the
benchmark to compare hardware it is necessary. If you use it to
compare SW service packs it is not necessary but SPEC won't accept
results.

TOP

 

It will be interesting to compare results from "real-world" users.
Please include the resolution that you're running at if reporting
results. Standardized tests are always run at 1280x1024. Running at
higher resolutions will result in lower scores - but may be more
realistic, too.

 

Which just goes to show that SPEC is a graphics benchmark.

TOP

 

Here are my test results, run at 1280x1024:

SolidWorks 2007 Workstation Benchmark
User Name : Art
Computer Name: M70
Manufacturer : Dell Inc.
Model : Precision M70
OS : Microsoft Windows XP Professional
OS SP : Service Pack 2
CPU : Intel(R) Pentium(R) M processor 1.86GHz
# of CPU : 1
Memory : 2047

**** Overall Test Results ****
Note: All results are in seconds.
Lower scores are better.

Test Number 1
Test Total = 306.56
Graphics = 101.71
CPU = 86.77
I/O = 118.08

Test Number 2
Test Total = 322.02
Graphics = 102.82
CPU = 85.69
I/O = 133.51

Test Number 3
Test Total = 313.28
Graphics = 101.21
CPU = 81.79
I/O = 130.28

Test Number 4
Test Total = 301.9
Graphics = 103.41
CPU = 81.42
I/O = 117.07

Test Number 5
Test Total = 301.56
Graphics = 105.44
CPU = 81.15
I/O = 114.97

Test Averages for 5 tests(s).
Test Total = 309.06
Graphics = 102.92
CPU = 83.36
I/O = 122.78

 

Compared with the 2005 benchmark your processor scores are awesome
unless you somehow got graphics and CPU mixed.

TOP

 

Here is the cream of the crop on the 2005 Benchmark. Apparently CPUs
and IO have gotten a whole lot faster and graphics cards have gotten a
whole lot slower.

SPECapcSM SolidWorks2005 Benchmark Result

Dell Precision Mobile Workstation M90 2.33GHz
NVIDIA Quadro FX 2500M
Submitted by: Dell, Inc.

Test Number 1
Test Total = 447
Graphics = 88.75
CPU = 183.03
I/O = 175.22


Test Averages for 1 tests(s).
Test Total = 447
Graphics = 88.75
CPU = 183.03
I/O = 175.22

Dell Precision Workstation 690 3.00 GHz
nVidia Quadro FX 3500
Submitted by: Dell, Inc.

Test Number 1
Test Total = 325.68
Graphics = 64.96
CPU = 132.23
I/O = 128.49


Test Averages for 1 tests(s).
Test Total = 325.68
Graphics = 64.96
CPU = 132.23
I/O = 128.49

 

My report was a cut-and-paste of the text file created by the test.

I don't have any previous results for comparison, mainly because I never thought the
SPEC tests related directly to real CAD usage.

Art

P.S. The graphics and CPU fans spooled up higher than I've ever heard them go. It didn't
sound like a Dell product -- more like Pratt & Whitney.....

 

I am trying to make a couple points about SPECapc for SolidWorks.

1. SPECapc is really not what you would call a benchmark in the
strictest sense of the word. Maybe a relative benchmark is as far as I
would go. You can't compare results from one year's benchmark to
another. It doesn't run reliably across different releases of SW. I
can't run the SPECapc2007version on SW2003. And if I run SW2007 on
SPECapc 2005 there will be the overhead of file conversion.

2. SPECapc does not reflect real world. If you look at the CPU scores
between 2005 and 2007 you will see a tremendous difference. Likewise
the graphics scores. If you take a 2005 assembly, one that you
struggled with in 2005, convert it to 2007, do you see the same level
of improvement that SPECapc shows on the new hardware you are
testing?

3. SPECapc does not reflect the real world in that it is heavily
biased to graphics card performance. Just visit the results page on
their site or look at the source code. Years ago when SPEC was a hot
item on the NG I did a lot of testing with it. I found that a good
graphics card could make a mediocre CPU look stellar and a bad
graphics card would make a good CPU look like a dog. In the real world
you can work around a slow graphics card by setting SW graphics to
their lowest settings. Sure circles will look like hexagons, but work
can get done. You can't work around a poor performing CPU and that is
where a lot of time is spent waiting on large assemblies and complex
parts.

4. SPEC does not benchmark a significant subset of SW functionality.
And it doesn't go out of it's way to test the functionality in SW that
causes bottlenecks.You won't see a lot of complex large assembly
drawings or assemblies with lot's of mega-multi configuration parts in
them. A lot of it is concerned with graphics eye candy.

5. Given that SPECapc 2007 is just out and that 2008 is now in beta it
has a limited lifetime so that in six months it will be relegated to
just another curiousity.


Here is a list of uses for SPECapc:
Home hardware evaluation
Business purchasing evaluation
Internal hardware development
Internal ISV development
User system evaluation/optimization
Research study
Vendor competitive analysis
Magazine or online publication

In other words you can't use it to evaluate one version of SW against
another.

I should mention that you hit on one of the strengths that SPEC has
always had, the ability to really stress a system. Any system that
won't run through all five iterations of SPEC probably has problems. I
have used SPEC in the past to evaluate registry tweaks. The newer
versions aren't as handy because they take so long to run. It is the
first thing I would run if hardware problems are suspected.

Why SW raises the CPU temperature so much still is a mystery to me.
But it does and more so that other types of software like FEA.

TOP

 

This one I think I know. FEA is almost entirely floating-point math and
block memory transfers, which only exercises those areas of the chip. SW
will exercise the integer and logic portions of the chip as well and
probably work the caches harder, thereby causing another portion of the CPU
die to generate heat. This will be more pronounced in recent chip designs
that can lower the clock speed or even switch off unused parts of the chip.
Currently, Intel's Core processors are the most agressive about this. It's
a significant portion of their power saving (in addition to the 65 nm
process, etc.).

 

Anna,
Here are my results from SW2007 x64 SP2.2 on an Asus M2N-SLI Deluxe mobo
with Quadro FX1500:

OS : Microsoft(R) Windows(R) XP Professional x64 Edition
OS SP : Service Pack 2
CPU : AMD Athlon(tm) 64 X2 Dual Core Processor 4600+
# of CPU : 2
Memory : 2046

**** Overall Test Results ****
Note: All results are in seconds.
Lower scores are better.

Test Number 1
Test Total = 344.65
Graphics = 94.49
CPU = 102.05
I/O = 148.11

Test Number 2
Test Total = 350.02
Graphics = 100.17
CPU = 104.85
I/O = 145

Test Number 3
Test Total = 396.4
Graphics = 107.95
CPU = 104.68
I/O = 183.77

Test Number 4
Test Total = 331.93
Graphics = 99.97
CPU = 95.51
I/O = 136.45

Test Number 5
Test Total = 345.76
Graphics = 92.98
CPU = 103.02
I/O = 149.76

Test Averages for 5 tests(s).
Test Total = 353.75
Graphics = 99.11
CPU = 102.02
I/O = 152.62

Cheers,

Gil

 

Anna,

I was being a bit sarcastic about SPECapc because it should be obvious
that CPU and IO should not flip flop with graphic scores. If SPECapc
was really a benchmark scores from one year would be comparable with
scores from another year. They are not.

TOP

 

Anna,

I use it for the same things you do. I just question using the term
benchmark because benchmarks aren't supposed to move and SPECapc moves
from release to release and year to year.

Call it a stress test or something else because that is all it is.

TOP

 

This is a three year old machine taken right offline from normal work.
Only the AV software was shut down.

SW2007 SP3.1
SolidWorks 2007 Workstation Benchmark
User Name : pbk
Computer Name: PBK-040505
Manufacturer : D.S. Electronics
Model : WC64
OS : Microsoft Windows XP Professional
OS SP : Service Pack 2
CPU : AMD Athlon(tm) 64 FX-53 Processor
# of CPU : 1
Memory : 3406

**** Overall Test Results ****
Note: All results are in seconds.
Lower scores are better.

Test Number 1
Test Total = 228.38
Graphics = 99.64
CPU = 44.02
I/O = 84.72

Test Number 2
Test Total = 224.37
Graphics = 98.1
CPU = 42.34
I/O = 83.93

Test Number 3
Test Total = 226.36
Graphics = 98.61
CPU = 43.31
I/O = 84.44

Test Number 4
Test Total = 226.45
Graphics = 98.38
CPU = 43.27
I/O = 84.8

Test Number 5
Test Total = 225.24
Graphics = 97.6
CPU = 41.99
I/O = 85.65

Test Averages for 5 tests(s).
Test Total = 226.17
Graphics = 98.47
CPU = 42.99
I/O = 84.71

 

Dan Bovinich has put together a spreadsheet of SPECapc results for SW
on Google spreadsheets. This link below should take you there.

http://spreadsheets.google.com/ccc?...=&t=7769282976297044622


TOP

 

Paul,

This takes me to a login page for Google docs & spreadsheets. I am loath to
setup a Google account just to look at the spreadsheet. Anna's link to her
spreadsheet in "New computer justification" put me straight to the
spreadsheet.

Jerry Steiger
Tripod Data Systems
"take the garbage out, dear"

 

Google Spreadsheets offers an option to "Publish" the spreadsheet.
Try this link and see if it works.

http://spreadsheets.google.com/pub?key=pgm3-ErF0HSyXJojlVfqEAg&inv

Notice the pub instead of ccc in the original link.

Regards,

Anna