PWx-2 memory issue

We had some big renderings to do- the models alone take 1.1 Gig to load -
so we purchased more memory to upgrade a couple of machines to 3Gig.

When we render, enabling memory usage to 6000K (6Gig, right?), the
renderings go for a little bit then crash. This has happened to us during
'indirect illumination' calculation, though we have it as low as it ccan go,
and I suspect strongly that its a red herring. After a little analysis with
task manager running during the rendering, we found that SWx/PWx crashes
(with a handy warning about 'running out of memory') pretty consistently
when the total memory usage eclipses approximately 1.7 Gig.

If we change the ray-trace depth setting so it is lower (around 35), we can
render without crashing. The max memory usage when it doesn't crash is less
than the magic number of 1.7 Gig (actually around 1.5)
We can also prevent crashing when we set the memory usage to 500K, so PWx
will dump its cache well before we get to the 1.7 gig number.

This does not seem to be a memory leak issue - the crashes happen on a fresh
reboot, as well as when SWx has been up after a few minor test renders on
individual components.

We have two different machines with different hardware configs, so I am
holding back on the basic machine date (though I can cough it up if it is
absolutely required). What I am really interested to know is:

1) Has anyone done a SW2004, PWx-2 rendering that used more than 1.7 Gig of
memory, be it physical or pjysical+ page file?
2) What were your settings if you were able to get it to use more than 1.7
Gig?
3) Any ideas on why we get a fairly consistent crash result when we eclipse
approximately 1.7 Gig?
4) Any ideas on why we are crashing, due to a lack of memory error, well
before we even use up the PHYSICAL RAM, let alone start hitting the hard
drive for virtual memory?

Thank you for any ideas or insight
-Ed

Note - I think the reason it is 'approximately' 1.7 gig is that other
programs that are running. If I have PhotoShop up, I can get up to 1.8 gig
before the crash. If nothing else is running, it crashes quite consistently
at 1.67 gig. I believe there might be an upper limit to what PWx can do -
physical or virtaul ram being irrelevent - but I am open to suggestions.

 

we purchased more memory to upgrade a couple of machines to 3Gig.

Have you enabled the MS 3 Gig switch? Note: the application
(PhotoWorks) must support the 3 Gig switch. I do no know if PWX
supports it. The 3GB switch allows a process to access up to 3GB
instead of the normal 2GB.

Without the 3 Gig switch enabled, the most memory any single process in
Windows can allocate is 2GB. This includes physical and virtual memory.
Note: your 1.7GB limit is pretty close to the 2GB hardwired limit.
Perhaps you are actually hitting this 2GB limitation.

Background:
For any 32 bit OS (including Windows), the maximum amount of memory that
can be addressed at one time is 4GB (2^32). The Windows OS always
reserves half of all available memory for itself. This means that if
you have the maximum of 4GB of memory available, 2GB is automatically
reserved for Windows and the rest (2GB) is available to any single
process. The 3GB switch changes this relationship, reserving only 1GB
for the OS and 3GB for other processes (this is the default in Linux).

Here is where it gets a bit fuzzy for me: You can have more than 4GB of
memory (combined physical and virtual), but each process must stay
within its 2GB (or 3GB) limit.

 

Hi Ed,

We ran into this 1.7GB limit using W2KPro OS when SW2003 was introduced.
SW in Mass. sent our assy's over to England where they were able to open
them on a server platform allowing the full 3GB of RAM to be used. I
have been told that WXP Pro has eliminated this restriction but have not
checked it out personally. The MS3 switch might be the fix. Do any group
members know if this should be applied to WXP Pro or is it part of SP1?

Regards,

Dennis Deacon

 

Possible work-around...

Export your part/assy via parasolid and bring it back in. Should shrink file
size considerably, leaving extra memory for the rendering? Of course you
lose all of the PW materials that you have set up so far so maybe not
feasible for your situation...

Todd


"Edward T Eaton"

 

Thanks to all who responded. I have blown a good half a day on this, but
boy have I learned a lot.

First, I need to ask if one of you could confirm that I have entered the
text correctly to enable the 3 gig switch, because the Microsoft web page is
miserable - it took forever to find the exact right phrase to search for the
switch information, and even then there is no explicit direction (that I
could find anywhere) on how to really enable it. Even SWs advice on this
subject, buried on page 7-12 of the 2004 whats new manual, points to
wonderfully imprecise pages at microsft.com that do not give exact, clear
instruction. Why can't they just say - to turn on the3 GB switch, type
[this]????).

Anyway, when I edited the boot.ini file, I typed the line '/3GB' between the
lines '[boot loader]' and '[operating systems]' lines. Is that how I turn
on the switch?

Now the bad news. If I did this correctly, it doesn't make a damn bit of
difference to preventing crashing. Actually, thats not entirely true - I am
now crashing at 1.6GB of memory usage instead of 1.67, but I think that's
splitting hairs. I do not see evidence that PWx can utilize the memory
freed up with the 3GB switch.

What does help is to LIMIT the 'maximum memory allocation' in PWx.
I always assumed that we should INCREASE the value in order to prevent PWx
from crashing - after all, the crashes say that PWx ran out of memory, so
more memory is better, right? But hey, why should intuition enter into
this.
If I look at the memory usage before I start rendering (1.2 GB for this
assembly), then subtract that from my crash point of 1.6 Gig, I have 400 Meg
left for PWx. If I set my maximum memory allocation to 350 (I want 50 Meg
of slop just in case), PWx builds up to 1.55 of memory use then releases a
little to go back below that set value. No crashes.
I think I'm pretty close to the right idea, because if I increase the
allocation to 450, I crash almost immediately. Thanks to the guys at CATI
tech support for pointing towards this novel approach.

In summary: I think the best practice going forward will be as follows:
Set the ray trace depth to a middling value - 35 seems to work well and not
crash every time (note for Paul Salvador - a higher ray trace depth is
supposed to cache more info and speed up rendering time). On smaller
assemblies or single parts, I will jack this all the way up.
Then I will use the 'maximum memory allocation' as a safety net, in case the
cache gets too large. This ought to be set per assembly, based on the
static memory load of the assembly in SW.

I am also going to have a little conversation with the folks at SW, because
this whole thing could have been avoided if they just limited the 'maximum
memory setting' so it couldn't be higer than 1.7GB - which appears to be PWx
limit, not even counting the usage of SWx which would nibble 100 Meg off of
that value. We wouldn't have blown a wad on more memory, and I wouldn't
have blown a morning experimenting with this crap, if they hadn't dangled
the (apprently) erroneous notion that PWx can use up to 10 gig of physical
and virtual memory to speed up rendering..

 

The application must support the 3 gig switch. I remember a statement
that SWX2004 now supported this memory option, but that does not
necessarily mean PWX also supports it.

Note: I have never used or needed to use the 3 gig switch, yet, but I am
getting close!

 

Paul, thanks for the link. I don't know how many f*(&($^ing articles I
looked through trying to find the simple example text that you provided in
the link below.

Of course, the problem is that my boot.ini file does not look like the one
they show. Mine has this '/fastdetect' element in it that I assume exists
for a reason.

Here's my boot.ini file:
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /fastdetect

Since we had nothing better to go on this morning, we originally tried:
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
/3GB
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /fastdetect

Nothing seemed to happen either way after this change. PWx still crashed,
but everything else seemed normal.

Then, Paul's link pointed us in the right direction. But since I was loathe
to delete that 'fastdetect' thing that must be in there for a reason, we
thought we'd give this a try:
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /fastdetect /3GB

Ummm, for the record, don't do that. Windows will not reboot ever again,
you can't get in to fix the file, and the machine dies.

Fortunately, we recovered by booting off of a copy of my operating system on
a spare drive, fixed the boot.ini file back to what it was originally, and
are ready for another round.

So, unless I hear otherwise, I will delete the '/fastdetect' bit and replace
it with '/3GB'. What the hell. Tonight, I make a backup of my C: drive
just in case this screws up everything. And I am quite curious why there is
'/fastdetect' in the file in the first place, though - any ideas?


This computer stuff is fun!

Ed

 

Page 7-12 of the what's new in 2004 manual says SW supports the 3 gig
switch. I have some feelers out about whether PWx enables it, and I'll let
y'all know if word comes through either way.

Begin editorial comment:
I wish they didnt call this the /3GB switch. A switch is a simple thing -
you flip it, and something goes on. This feels more like the /3GB hack,
where you have to take a delicate system file and, with very little
information, figure out the correct code and syntax to successfully hack
your system. 'Switch' my ass.
End editorial comment.

 

Ed,

I'd be very careful about putting a switch that close to my ass. The first
definition in my dictionary for switch is "A slender flexible rod, stick, or
twig, especially one used for whipping".

Jerry Steiger
Tripod Data Systems

 

***Disclaimer-if you are sensitive about the exposure of faults in
SolidWorks software it is advised that you refrain from reading this post.
Reading this post may cause the following: you might cry if you make a
living off this software, get mad, agree with me, or in extreme cases
contemplate demoing Inventurds, Pro/Enema, Unigiggles, Solid Idiot or even
Cad Star 5D(still in Beta). Could even cause babbling about seamless,
unified, hybrid modeling and cause paranoia that the CAD monkeys are
brainwashing everyone into thinking they can get the job done via the 'hack
and whack' methodology and are constantly following you with 'white papers'.
The following information is not propaganda and is intended to share real
world frustrations with SolidWorks software. If you are employed by
SolidWorks, by reading this post you are hereby agreeing to publicly
recognize the issues with your software and make a public apology, service
pack and free subscription available within 30 days.***

All,

This is definately not contained to PW2 itself. This is a core issue in
SolidWorks and has always been there. They hurried up and wrote some code so
we could open up a 7,500+ part assembly 2 years ago. In other words 'hurried
up and put on a band aid' so we would buy 10 seats. SolidWorks is definately
aware of the problem, but I guess they a)don't really care or b)don't have a
company with 200 seats threatening to move to a different CAD program unless
it is fixed. SolidWorks Corp. has had my large assembly models for a year
now and the only response I got from them was 'hey we can open them'. Wow,
open them. Now thats really all we do out here in the real world. I wouldn't
be so bitter if they didn't advertise large (10,000+ part) assembly
performance.

We experienced the same problems in SW2003 and SW2004. Unfortunately, the
/3GB switch also does not work in OS's other than XP and 2003 server. I was
on Win2000. What kills me is they brag about 'unsurpassed large assembly
functionality' and 'we have a user with a 100,000+ part assembly.'
Blah-Blah-Blah. I dealt with the crap for 2 years and fortunately for me I
now work at a company that does small assemblies. I'm also proud to say that
I had to take screen shots of an assembly and paste them into a drawing
because SolidWorks(sometimes) would die otherwise. Maybe thats what they guy
with the 100,000+ part assembly is doing. Try cropping a large assembly view
and see what happens Mr. large assembly guy.

The out-of-resource issues for will show up for everyone around 1.6-1.7GB.
My personal opinion-ain't gonna get fixed anytime soon. I mean they have
people like Ed Eaton running all over the MS knowledge base all day just
trying to figure out how he can hack his boot.ini file. And Ed , c'mon, you
know SolidWorks wouldn't explain how to do it in any of their documentation.
That would mean that if you followed the documentation and it didn't work
they'd be at fault. They've been poiting the finger at MS since I brought
the issue up.

 

Funny - we did the dos boot disk thing, but could not get back to the C:
drive to resurrect things. It was stuck on A:
Just a warning to those of you who want to try this at home. Scary stuff.

 

Thanks Dale - I'll make a note of that. So your saying that my boot drive
has to be FAT32, right?
We have another way that is pretty darn safe, and won't require me to
reformat anything. We have removeable hard drives, and I just copy my C:
drive to the removeable before making big changes (nice to have a really
huge safety net). When everything goes to hell, we can boot off of the copy
and, once booted, we can fix just about anything. Its nice to have the copy
in my desk drawer for other reasons - if the boot drive goes bad, we get hit
by a virus, or any of the other digital plagues nail us, we can plug in the
backup and keep working with barely a hiccough.

 

Just for the record:
I changed the boot.ini file to match the text on the Microsoft page that
Paul linked us to.

timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /fastdetect /3GB

That killed my system again. End of this phase of the experiment. I'm
starting a new thread, and we cn maybe continue the fun there.

Thanks to everyone who pitched in!

 

Jeff,

Thanks for the warning.
I just finished a large assembly, but only got to the 1.4 gig level and that
was on the master assembly drawing. I stripped the drawing so as not to go
any higher.

We then purchased 3 gigs of ram. I never did exceed the 1.4 gig level and
am now not to interested in having to do larger assemblies.

I may have another large assembly to do real soon (meeting in Detroit
Friday). I'm going to set up the procedures for handling large assemblies
for the upcoming project and I sure would like to get any advice you might
have as to how you handle large assemblies and what all of the options might
be.

Tom Chasteen

We can take it off line if you prefer. Thanks in advance.

 

FYI - with the correct syntax, and a hotfix from Microsoft, the 3GB switch
does work and will keep PWx renderings from crashing when it hits 1.7GB of
memory usage. See the new thread on the /3GB switch for details on what I
had to do to get it to work.

 

Thanks Ed.

I've been following the /3GB thread and will check the new info.

Tom