Carrige Return

Blitz3D Forums/Blitz3D Beginners Area/Carrige Return

bushsolo

(Posted 2003) [#1]

I have a CSV file which I want to feed into an array in Blitz. I need to be able to detect carrige return since that would mark the next line of data.

I have exported information from a database and need to read each line as a new line of data. I have 200 lines each with 10 values, I could just us a 200 by 10 array but this amount of data and lines will change so I need it to be dynamic. As in it will need to read the amount of commas until the end of line then that will tell me the amount of data each line will have.

TIA

Odds On

(Posted 2003) [#2]

ReadLine() will read a single line of text at a time.

While Not Eof( File )
    LineData$ = ReadLine( File )
    ;Parse data on this line
WEnd

soja	(Posted 2003) [#3]

As Fullernator points out, ReadLine$ would probably be your best bet (and the easiest).

For future reference, Chr$(10) represents Line Feed, and Chr$(13) represents Carriage Return.

bushsolo

(Posted 2003) [#4]

Good stuff,

while ReadLine seems very useful I'm still going to read the file in byte by byte since I need to check for the comma while I'm taking the data in rather than sort the line out after.

chr$(13) - Seems like the solution, hope it works.

soja	(Posted 2003) [#5]

Remember that in CSV files (at least Windows CSV), the end-of-line marker is a CR+LF, not just CR or LF. So you'll have to make sure to parse your stuff right.

E.g.:
blah,blah,blah[CR][LF]blah,blah,blah

Odds On

(Posted 2003) [#6]

To parse the data you might find these functions useful (I swear by them)

Function CountItems( Message$, Sep$ = "," )

	For A = 1 To Len(Message$)
		If Mid(Message$, A, 1) = Sep$
			Count = Count + 1
		EndIf
	Next
	Count = Count + 1
	Return Count

End Function

Use this function to count how many items there are on the current line.

Function Parse$( Message$, Item, Sep$ = "," )

	I = 1
	Repeat
		Txt2$ = Mid(Message$,I,1)
		If Txt2$ <> Sep$ Then
			Txt3$ = Txt3$ + Txt2$
		Else
			If Pos = Item Then
				exit
			Else
				Pos = Pos + 1
				Txt3$ = ""
			EndIf
		EndIf
		I = I + 1
	Until I > Len(Message$)
	Return Txt3$
	
End Function

Then use a loop with this function to get the data.

Here's an example:

txt$ = "one,two,three,four,five,six,seven"
NumItems = CountItems( txt$, "," )
For A = 0 to NumItems - 1
    item$ = Parse( txt$, A, "," )
    ;Do something with the item
    debuglog item$
Next

bushsolo

(Posted 2003) [#7]

Fullernator well done.

These function turn out to be just what I'm looking for, instead of doing it byte by byte I'm now reading through the lines one at a time. This helps it be dynamic, the user would not need to enter the number of values on each line.

One thing I'm still stuck on is I have to predifine the size of the array. Currently 200,12 handles it but I would like this to be dynamic. With the CountItems function I can get the second array value of 12 but I need to find out the number of lines in the CSV file for the first array value.

Odds On

(Posted 2003) [#8]

You can do it by looping through the file before you read the data, and then going back to the start of the file.

file = ReadFile( "file.txt" )
NumLines = 0
While Not Eof( file )
    nothing$ = ReadLine( file )
    NumLines = NumLines + 1
WEnd
SeekFile file, 0 ;Go back to start of file
While Not Eof( file )
    ;Read the data
WEnd
CloseFile file

Now that I think about it, you could get the second array value at the same time by using CountItems() on nothing$ and seeing if it's larger than the current value.

bushsolo

(Posted 2003) [#9]

Seems like a simple enough solution, my only fear was a slow down as it counts line.

Thanks again, I know I'll be back

myspys

(Posted 2003) [#10]

bushsolo, one way of doing it could be to read the whole file into a bank or into a string, and then count the number if CR/LF's..

Oldefoxx

(Posted 2003) [#11]

You can sometimes consider using two arrays. You define one to a set size, such as 100. when you reach that max size, you define the second array to a larger size, say 200, and copy the first 100 elements from the first array to the second. Then delete the first array. If the second fills up. You dimention the first array again of a larger size, perhaps 300, and copy the second array back into the first, then delete the second array. There are certain disadvantages in this approach, but it is hardly worse than some of the other techniques suggested.

A slightly faster way is to define a new array for every 100 entries. if you end up with an index of 534, as an example, you know that it is the 34th element in the fifth array that was created. But depending upon the nature of your code, this may prove to be unsuitable for your needs.

If you do not need to have all the lines in memory at any given time, you could take the original file and read through it, and just track the current location in the file where each new line starts. You might then have an entry for the 534th index of 23142, which means that the 534th line in that file starts at position 23142. One you create the index reference to the file, you could write it as a separate file to the hard drive, then read it back in as a fast means of quickly accessing the original file in the future. There are a lot of possible ways to handle files with lines of data in them. Assuming that you have a file called input.txt, then you could create an index file for it called input.idx. The fact that the names match, except for the extension, tells you that the .idx file is an index for the same name file that has the .txt extension. Understand, this is merely a technique available to you - it does mot mean that if you find an .idx file in some other context, that it indexes any .txt file, or that .txt files all come with index files. Things like this are subject to your own determination and efforts.