ASSURA LVS does not finish.

Hello Experts,

We use ASSURA for DRC/LVS and extraction. Entire setup was working
perfectly until last week, and all of a sudden LVS does not work.
Basically LVS stops in the middle and does not finish. I am trying to
look for loose ends, but I dont get much information in the log file. I
see that the process launched to complete the LVS has entered the 'S'
mode when I check the 'ps -la' output. Basically, ASSURA LVS log file
stops at:


-----------------8<------------------snip --------8<-------------------
.....
......
.....
run on "server name" from
/disk/cadence/assura/tools.sun4v/assura/bin/32bit/dfIIToVldb.exe on Tue
Sep 12 09:46:22 2006

Library Name: "tsmc18_char"
Cell Name: "assura_test"
Cell View: "schematic"
Output Data Base Name:
"/home/kulkarni/tsmc_design/assura_runs/lvsruntest.sdb"
Simulator Name: "auLvs"
View List: "auLvs schematic symbol"
Stop List: "auLvs"
Net Listing Mode is Analog

-------- LOG FILE STOPS UPDATING HERE. -------------------

I checked for ps on dfIIToVldb.exe and it has entered sleep mode. There
are no other errors. But LVS does not finish. Is there any other
information that I can look for which will tell me what is wrong??????

Thanks for your help. Its greatly appreciated.

Regards,
Raghavendra

 

Raghavendra,

Ouch!

I have a vague recollection of some kind of lock file in the Assura
run directory sometimes causing trouble. Try running in an empty
local diretory (like /tmp). Try removing and re-creating your
assura_runs directory. Try 'truss -p pid' on the hung process.

I'm assuming you're running Assura from the Virtuoso GUI. If so,
get set up to run it from the command line instead. You just need
all the same path entries and environment variables you already
have set up for Virtuoso. License server, ASSURAHOME, path,
analog netlisting mode, etc.

Then just 'assura .../assura_runs/lvsruntest.rsf' should work
(well, it should fail in the same way). Then you truss -f that run:

% truss -f assura ..../assura_runs/lvsruntest.rsf >& /tmp/trussout &

....wait until that hangs, then start searching backwards through
the system call output.

-Jay-

 

Hello Jay,

Thanks very much for the help. I followed all your instructions - run
assura from command line with fresh directory (copied .rsf and .vlr
files from previous runs). Debugging reveals that the processes that
are SLEEPing are "avlck" processes.

6500 pts/7 S 0:00 /disk/cadence/assura/tools/assura/bin/32bit/avlck
/disk/cadence/assura/tools/assura/bin/avlck PAT

I cant get 'ps' to print more than 80 characters for the command
options and cant figure out which file its waiting for lock.

Looks like its basically running: avlck on avlck!!! Can there be any
locks generated in the cadence installation directory?????? Any
help/tips to identify the cause of the error woudl be greatly
appreciated. Thanks for your help.

Regards,
Raghavendra

Tail of the truss output looks like.

--------- Output upto here is expected ...or as seen in log file
----------

Net Listing Mode is Analog
6508:
readlink("/disk/baby/cadence/ic5033/tools.sun4v/dfII/etc/cdslib",
0xFFBEA4D8, 1024) Err#22 EINVAL
6508:
readlink("/disk/cadence/ic5033/tools.sun4v/dfII/etc/cdslib/basic",
0xFFBEA4D8, 1024) Err#22 EINVAL
6508:
readlink("/disk/cadence/ic5033/tools.sun4v/dfII/etc/cdslib/basic/iopin/symbolr",
0xFFBEA4D8, 1024) Err#22 EINVAL
6508: stat64("/etc/mnttab", 0xFECC8748) = 0
6508:
open("/disk/cadence/ic5033/tools.sun4v/dfII/etc/cdslib/basic/iopin/symbolr/master.tag",
O_RDONLY) = 106475: Received signal #14, SIGALRM, in read()
[caught]
6475: read(0, 0x000230F4, 5120) Err#91 ERESTART
6475: getpid() = 6475 [6473]
6475: sigaction(SIGALRM, 0xFFBEF340, 0xFFBEF3C4) = 0
6475: alarm(20) = 0
6475: setcontext(0xFFBEF4E8)
6475: read(0, 0x000230F4, 5120) (sleeping...)
6500: Received signal #14, SIGALRM, in read() [caught]
6500: read(0, 0x000230F4, 5120) Err#91 ERESTART
6500: getpid() = 6500 [6498]
6500: sigaction(SIGALRM, 0xFFBEF340, 0xFFBEF3C4) = 0
6500: alarm(20) = 0
6500: setcontext(0xFFBEF4E8)
6500: read(0, 0x000230F4, 5120) (sleeping...)

......Repeats.....
......Repeats.....

 

Has any one had this problem with avlck and lock files when using
ASSURA? LVS just does not finish even when run in a clean directory.
Any more thoughts on what loose ends to look for??

Thanks!
Raghavendra

 

Which might not actually be a problem. IIRC, in normal operation
avlck just hangs around waiting for the run to finish, clears the lock
files (or waits for them to be cleared, not sure) and exits.

This looks normal enough, standard dfII library code, checking
everything in the universe multiple times to see if it's a symlink.

That looks wild. That return value from open can't possibly be
a valid file handle, can it? It's supposed to be either the
lowest available filehandle number and I don't think
"106475" can qualify. And who's issuing SIGALRM?

....and then we try to read 5120 bytes from fh 0 (almost always
stdin) and fail with ERESTART (try again after interrupted by
signal). Sorry, no idea what this is doing. Smells like a bug.

If possible, update the installation to latest ISR and hope.
Sorry not to be more help.

-Jay-

 

I dont know what is the cause of the problem. But after many wasted
hours of looking though the logs and truss outputs, here is what we
did to solve the problem:

Cadence installation was an auto mount from another server in a
different subnet. This has been working for several years now. So I
believe some "new" change in the network could stop "lock" services in
cadence lock files which was causing the problem. We copied the entire
installation directory to a local server in our subnet. Remounted using
nfs (instead of auto mount). This for some reason solved the problem
and now ASSURA runs ...and does not hang in the middle.

Any ideas, why this solved the problem? and what change in the network
would cause this error.

Thanks!
Raghavendra