1,057 Bugs Found in SolidWorks 2004

http://www.d-digest.com/soliddigitaldigest/v4i5/usernews.html

Chris Garcia, VP of R&D at SolidWorks also cited some impressive QA
statistics at the press dinner Monday night. SolidWorks worked with
more than 3,600 users during beta testing for SolidWorks 2004. There
was a competition to see who could find the most problems. After 4,776
hours of usage, a total of 1,057 verified bugs were identified. Garcia
said that 80% of those bugs were fixed before first customer ship of
the new release on September 8, 2003.

Incentive prizes included a HP workstation and several i-Pods, with
users earning points according to how many bugs they discovered. The
top five performers by points were:

Jason Caprioti: 445
Stefan Berlitz: 393
Scott McFadden: 392
Casey Kimes: 233
Clarence Ivester: 231

Garcia said a similar beta test program for the next release will
being in May. The goal is to capture 2,100 bugs – twice as many as the
previous release.

2100 bugs?! So are they planning on this next version to be worse than
2004?

 

Isn't every release worse than the release before it? I think it's their
methodology.

- Eddy

 

No question about it, the level of quaility control is slipping at
Solidworks!
To many bugs folks.

 

Ok, I guess I'll just accept the fact that there are people who have an
overwhelming sadistic urge to punish (or ridicule according to that
Armadiller feller) anyone who is successful, but makes mistakes (meaning
anyone who is successful).

I know it's very unfashionable to say anything positive here, but the
rest of the story, according to Cadalyst...
http://cadalyst.adv100.com/cadalyst/article/articleDetail.jsp?id=82543

===========
SolidWorks again plans a beta contest for SolidWorks 2005 beginning in
May. The beta contest for SolidWorks 2004 produced close to 4,000 bug
submissions by 3,600 contestants (1,057 unique bugs were verified). The
company says that thanks to this vigilance, the shipping version of
SolidWorks 2004 generated 28% fewer bug reports that its predecessor
SolidWorks 2003, even though its user base was 35% larger.
===========

People who say things are getting worse either haven't been around very
long, don't have a clue or have some other agenda.

Matt Lombard

 

Indeed.

The SHIPPING version. SP0.0.

Back when SW QA <sic> benefitted from real users testin the program.

And first release shipped without user beta testing resulted in a POS.

The problems start when we have to rely on SW to do the bug testing.

-nick e.

 

With all due respect matt, that's a complete crock of corporate group hug
BS. I've been using Solidworks full time since 1997 and I'm more than
qualified to say that things are getting worse. My agenda here at the
moment is to vent. No lies no surprises. I'm honest about that. But
what's your agenda? To temper me or anyone else who points out the truth?
Let's call a thorn a thorn and get past this. We don't write the software
or the SP's. We are not to blame when a bad release hits the street. We
should not be told to formulate work arounds when something that was sold to
us doesn't work properly. We know how to formulate work arounds. My clear
and present purpose is to say I'm getting tired of it. Do you want me to
jump ship more than you want your software to work properly? Is so, you've
got a mixed up set of priorities. When's the last time you were late for a
product release because a loft wouldn't work properly or because a sweep
that you did in a previous release inverted itself and you had to recreate
something? When the last time you spent 90 mins working on a part because
it took you an extra 30 mins of zooming and out to work around the clipping
problem?

The software is getting worse, and buggier with every release, due mostly to
new "features" that are of questionable use to most people, and I believe,
due to their development methodologies. I don't know for a fact where the
development is done, who is doing it, or how it's done. But my belief is
that marketing is prioritized over QA, and that's bad methodology. Fix
what's broken and move on from there. They will never again release a
reliable version if they keep this up. And that's not doing much with our
maintenance dollars. Improve surfacing, improve reliability, improve speed,
and then you can waste my time developing useless add-ins or non value added
"features". Yes, it's getting worse. And people who say things aren't
either haven't been around very long, don't have a clue or have some other
agenda.

- Eddy

 

After going off on matt about this topic, I collected the following data
from SW website. What do we make of this....

SW2003 up to SP2.0
1159 issues addressed (231 in SP0.0)

SW2004 up to SP2.0
388 issued addressed (163 in SP0.0)


So are we to believe that 2004 is less buggy or are they just able to
address fewer bugs now due to a new development approach? We saw recently
with SP2.0 being pulled that the latter may be true.

- Eddy

 

Matt:

Only 1.1 bugs submitted on average per beta tester? I was in the top 15 and
submitted a lot of bug reports. That means that a lot of the Beta copies
were simply toyed with or sat on someones desk unused. On top of it all only
1/4 of them were verified which means some 3,000 bugs were written off as
unreproducable. How many of those 1,057 have actually been fixed? These
numbers are far from being impressive.

2004 has been out a much shorter time than 2003, so of course there will be
at least 28% fewer bug reports. And don't tell me they are only counting
bugs submitted during Beta. Also 28% larger base? How many of those are
education related? I have a feeling the educational licenses hardly get any
bug submissions. Then you also have to factor in how many people are getting
fed up and not reporting anything as well. Again, these numbers are simply
Marketing hoopla.

I don't think any of us are punishing SolidWorks for being successful. We
simply demand good software. Hell, we're paying for it and many of us have
tried or are still trying to help them on their own time to make it better
as well. I don't think anyone is out of line saying things have gotten
worse.

It is painfully apparent that the SolidWorks community, at least those that
participate in this newsgroup and the SW forum, and those I know in the area
that use SW are wanting the software to perform faster and more reliably.
It's now SolidWorks chance to listen up and deliver what we want. I guess
what it comes down to for them is do they want to add shiney new features to
demo to new users or do they want to keep their user base satisfied? I'm
sure they've worked out the numbers or else the path they have chosen would
have been different. They'll probably gain more customers than lose based on
their current plan. And that's what it comes down to doesn't it?

As for me I'll still use SolidWorks, it's just now I have my eye more open
to alternative CAD packages.

 

Then your memory is short and inaccurate. SW2004 is the least crashy
version I have ever used. Interface bugs can usually be worked around
easily, but crashes are certainly the most productivity-robbing bugs,
and duly get the highest priority from SW. I do not have a single open
crash bug.

Looked more permanent than momentary, but I'll take your word for it.

My agenda is to have rational discussions about the software, where I
can learn things and help folks. No one benefits from personal
venting, especially when there is little else. There is such a thing
as talking about problems in a way that benefits users.

If you read my posts back a few years, you'll find that I used to post
sh_t like what you write. Maybe I grew up a little. Maybe I realized
that acting like an idiot marginalizes the idiot, not the idiots
target. Maybe I got tired of ranting against things that I couldn't
control and decided to be able to make best use of the tools regardless
of problems with the tools.

Ok, have it your way. What are you going to do meanwhile? You might
get a few bucks for it on eBay.

You flatter yourself. You don't have any affect at all on the software
working properly unless you are submitting bug reports and bugging your
VAR and regional technical folks to escalate issues.

Never. And that should never happen to anyone who is a self-proclaimed
"high end expert". The first thing a "high end expert" knows is 5 ways
to do everything so he can get his job done on time. People that work
with other software tools are going to tell you the same thing.

Ok, you've finally said something that I can agree with. Every company
that has products to develop has a balance to strike, and I agree that
I would prefer the balance was closer to the technical end than the
marketing. But like it or not, this ng is not a big enough tail to wag
the dog, because it's not just an R&D director, or SolidWorks Corp or
Dassault, or even the CAD industry, its all of software development.

Anyway, do what you can to be helpful, and lower your blood pressure a
notch.

matt

 

Let me summarize....

we agree on...
- Solidworks crashes to desktop less frequently then it used to (hmmm, maybe
but read on)

- Solidworks is putting more emphasis on marketing then on QA, true and like
the rest of the software industry (and in particular, the way Autodesk did
before people switched to Solidworks)

- Solidworks isn't going to change anytime soon. But that's no reason for
user complacency. By holding back maintenance dollars and communicating our
sense of disdain with our vars we will enact change... eventually. But
getting together about it does two things, it preserves sanity to know
you're not alone. And it convinces those on the fence that it isn't them
and that things are what they appear. It doesn't help to come in and tell
anyone who isn't waving a SW flag to calm down and formulate a work around,
we've already been down that road. It's not always the users matt, it's the
software and users have a right to bitch. Period.

we disagree on...
- Modeling bugs IMHO are just as much a set back as crashes. You relaunch
the software or you spend your time zooming in and out a million times to
see the edges that keep disappearing... pick your evil. Anything that
interrupts flow is flawed. And it's more flawed than ever thanks to the
interruptions in flow.

- I'll take your word for it, I'm sure I just missed the ones where you
vented, but a search on google going back to the beginning yields a "matt"
doing one of two things, either providing useful trouble shooting advice
which I seriously respect, or defending Solidworks, which was admirable in
the beginning but after the 2003 and 2004 fiascoes, it's just silly. Whoop
whoop hurray. You choose to use your power for good. Whatever.

- you come in here practically on a daily basis and defend SW when the issue
comes up. You used words like "idiot" to describe watercooler venting
behavior. For the record, I don't use the word "milk toast" "corporate
lackey" or "kiss ass" to describe those who think SW Corp should be
forgiven. It really makes me laugh when you calmly say it's wrong and
useless to attack SW yet you routinely attack the users who attack the
software.

- blood pressure here is fine. if you choose not to vent anymore that's
your call but look at the overall tone in the ng recently. if anyone is on
an island here, it's you. people are getting fed up. it's not personality
of the user, it's the insults from the product.

 

I have that bumper sticker, that was me!

 

Try this then:

In a drawing, select a dimension, then hit the expand arrow for layers.

There's your open crash bug. I even have an SPR for it.

I have one involving a cam-mate as well.

That's two I have.

Maybe you should try submitting more of your crashes as bugs and join the
rest of us.

How's about the time you have to waste redimensioning your drawings because
SW no longer correctly display dims for cuts that are defined as
offset-from-surface?

I don't care how much or little of an expert you are, I should be able to
print out drawings that I made 3 years ago without having to examine each
and every one to see if the latest SW service pack broke something.

Well, maybe engineering over at Dassault/SW needs to get a backbone and tell
marketing to shove it.

--nick e.

 

Ok, Ed, you win. You out-last me. My bitching at you and yours is as
hopeless as your brand of bitching. I'll at least stop before it gives
me a headache.

matt

 

No crash here.

Jim S.

 

maybe that's not the exact sequence of events then. I forget...maybe try
just clicking on the layer button? I don't know....it is reproducable and I
have an SPR for it. If I remember I'll try it when I get back to work and
see what the correct crash sequence is.

--nick e.