painfully slow string concatenation

Discussion in 'AutoCAD' started by Mark Propst, Jul 18, 2003.

  1. Mark Propst

    Mark Propst Guest

    Hi,
    Trying to get faster (text) file reading time.
    I was reading a series of text files, and concatenating their contents into
    a large string.
    this is unbelievably slow.

    if i just read them, 1096 files are read in less than a second so it's not
    the reading that's slow as I originally thought.

    putting the strings into a collection is also fast (seemingly immediate -
    need to get a timer working to check actual)

    is concatenating inherently slow? probably because internally a string
    array is being created and redimmed repeatedly?

    just curious
     
    Mark Propst, Jul 18, 2003
    #1
  2. Why are you concatenating strings, anyway?
     
    Tony Tanzillo, Jul 18, 2003
    #2
  3. Mark Propst

    ljb Guest

    String concatenation is slow. Here is an article on it and how to improve
    it. http://www.vb2themax.com/HtmlDoc.asp?Table=Articles&ID=30

    I also posted a CFileCopy class by Karl Peterson in news group
    autodesk.autocad.customer-files on 6/3/2003. Subject line was "PLT files".
    It is a class I use to concatenate hundrededs of plot files into one large
    plot job. For some reason Google doesn't show it in the search results.

    LJB
     
    ljb, Jul 18, 2003
    #3
  4. Out of curiousity, what about sizing your string one time and then loading
    the substrings into it using Mid() function. Does this example scale up
    well to your problem? I don't know if Mid() is a slow function or not.

    James

    Sub test_Concat()

    Dim a As String 'concatenated string
    Dim b(0 To 7) As String 'substrings
    Dim aLen As Long, curByte As Long 'length of final string
    Dim strings As New Collection

    b(0) = "abcde"
    b(1) = "fghijklmn"
    b(2) = ""
    b(3) = "opq"
    b(4) = "rstuv"
    b(5) = "wxyz"
    b(6) = "0123456"
    b(7) = "789"

    For i = LBound(b) To UBound(b)
    strings.Add b(i)
    Next 'i

    aLen = 0
    For Each curstr In strings
    aLen = aLen + Len(curstr)
    Next 'curStr

    a = Space(aLen)

    curByte = 1
    'I'm always nervous trusting For...Each to traverse the collection in the
    proper order, but
    ' I guess it does, because the alphabet is not scrambled.
    For Each curstr In strings
    Mid(a, curByte, Len(curstr)) = curstr
    curByte = curByte + Len(curstr)
    Next 'curStr
    Debug.Print "after: '" & a & "'"

    End Sub
     
    James Belshan, Jul 18, 2003
    #4
  5. As Tony asked: why are you doing this?

    Rather than read one line at a time and concatenating the results, you
    might try opening the file in Binary mode and reading a number of
    characters equal to the file length. If you prefer, you can use the
    ReadAll method of the TextStream class. You'll find the TextStream
    class in the Scripting Runtime library.
     
    Frank Oquendo, Jul 18, 2003
    #5
  6. Mark Propst

    Mark Propst Guest

    Thanks Frank,
    I'll try those also.

    see my reply to him - just my usual idiocy! :)
     
    Mark Propst, Jul 18, 2003
    #6
  7. Mark Propst

    Tim Arheit Guest

    You got it. Every time you concatenate a string a new one is created,
    the contents of both are copied into the new one, and any cleanup
    necessary takes place. Not bad for small strings, but with many large
    strings it becomes very slow. There are some options though...

    1. Load each string as one element of an array and use join to
    combine them.

    2. Allocate a big string yourself and place each substring into it
    yourslef. ie:
    s = space(10000)
    mid$(s,10,100) = tempstring ' Place tempstring at position 10
    mid$(s,110,200)=tempstring2

    -Tim
     
    Tim Arheit, Jul 18, 2003
    #7
  8. Mark Propst

    Mark Propst Guest

    Mark Propst, Jul 18, 2003
    #8
  9. Wouldn't Windows Find/Search work for what you need?
     
    Tony Tanzillo, Jul 18, 2003
    #9
  10. Mark Propst

    Mark Propst Guest

    it absolutely does work for what I need.
    I was trying to beat it's time in searching, and also be able to do it
    inside acad instead of opening or tabbing to explorer, browsing to the
    folder, filling out the fields, etc.
    besides, what's the fun of letting someone else do your work for you??? :)
    I guess to get the speed I'm thinking about I'd have to learn to write
    assembler? not in this lifetime!

    now that I see that vb is actually reading the files faster than lisp is, if
    I can rewrite all my lisp code that finds the functions into vb code, then I
    can test a vb approach.

    Except I have to use lisp to call vb and that always is really slow to call
    up a dvb from lisp, don't understand what the delay is but it's always been
    noticable waiting for a vba dvb to load and start. not sure if calling a
    dll will be faster,

    It may still probably be faster that starting explorer, browsing, entering
    fields etc.
    we'll see....
     
    Mark Propst, Jul 19, 2003
    #10
Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.