SWX 2003 Benchmark

whatever happened to the SPECapc benchmark for SWX2003?
It was due by 2nd quarter according to GPC News.

http://www.spec.org/gpc/

 

What would you like to see in a benchmark? The spec and the Solid Solutions
benchmarks are primarily testing the graphics card. In my experience the
graphics card is not usually the bottleneck.

My thoughts to get things rolling are:

A benchmark should have two parts:

1. An exhaustive test of each command broken down into user interface and
actual command performance.

2. An exhaustive test of real world modeling, perhaps broken down into part,
assembly and drawing performance (and perhaps geometry import should be
included too.)


In addition a benchmark should run without problem on pretty much any
version of SolidWorks from SW99 to the present in order to be able to
benchmark performance changes. The benchmark should report things like time
taken to complete a task and also resource usage. The benchmark should be
GPLed so that the source can be made available.

 

Paul,

Sounds like a good plan. It won't give anyone much to cover up with.

So....when are you going to have it finished ????


Regards


Mark

 

I would say such a project needs to be a group effort.

 

ND,

Not exactly. I don't think you can test the relative performance of
different versions (releases), on the same hardware. If you could you would
see a declining curve, from SW99 to the present, that is grossly out of
proportion to the added overhead of the extra features. SW doesn't want you
to be able to assign a number to this. They're counting on faster hardware
to cover this up.

Slash and burn programming at it's best


Mark

 

I've looked at the source code which is open source. There is one little
section of code that tests performance and it is hardly exhaustive. And the
algorithm they use to calculate the final result is heavily biased towards
graphics. This is not surprising because the people with the most input are
graphic card suppliers like 3D Labs.

Just because the test has a CPU report in it, don't be misled in thinking it
is at all exhaustive.

 

Mark,
I think you have a point here. I like the graphics check but would
rather see the CPU tested more.

Certainly their software has slipped in performance; the drawing
package has gone backwards

 

Snip

Snip

Paul,

I would love to see such a benchmark, and agree that a group effort is the
only reasonable way to get there, but I do see a problem with your
description. It can't work back to SW99 and test every command, since some
of them were added after SW99.

I could see a set of benchmarks. One for the flat plates with holes people,
one for the surface crazies, one for the lots of parts people...

The first and third probably would work fine back at SW99. The second would
probably have to have slightly different parts for each release.

Jerry Steiger
Tripod Data Systems

 

Jerry,

You are right about not every command being available that far back. And
some commands change from release to release.....what is it, OpenDoc4 by
now. One way to handle that is to use the old version API call and the
other way is to test for the release of SW and execute the correct API
calls for that version. Perhaps being somewhat limited is a good thing.
After all, one purpose for a benchmark is to keep an eye on the core
functionality that is already there and not be distracted by all the new
gee whiz stuff.

Having benches that test various subsets is a good idea. Of course it will
require people to think when interpreting results.

I have a couple of the prismatic part routines already built. One does holes
in a plate and other creates a fractal in solids. The hole in a plate
routine is capable of running almost any Windoze box out of memory.