Index Searching large text files

[hitman]

I created an index for two large text files, process completed successfuly with no errors in the log. When I go in and search the index for a word, such as Warning* that I know are in the logs several times it comes back as no results found. I have gone into the Advanced settings and tried both Any search words and All search words, still the same results.

I am running 0.99j on a Windows 7 64bit.

Thanks.

[David (PassMark)]

Did the files have a .txt file extension? Or where they of some other type. (e.g. .log, .html, .dat, etc..)

Did you check in the index log that the files concerned actually got indexed. There should be a line for each of your text files like,
Indexing C:\foldername\filename.txt

Do you get any search matches if you search for just, Warn*

In the Manage case window, can you right click on the index and check how many Bytes and Unique Words were indexed and see if the numbers sound about right for the files.

How big where the files. There is a size limit per file (adjustable from the advanced index settings). I think the default size if around 200MB. So they would need to be big text files to hit this limit.

If you still have a problem can you zip up one of the text files and E-Mail it to us.

[hitman]

Yes both files had the .txt extension. One of the files was over 600MB, I did try adjusting the settings to allow for a larger size file, it allowed me to set it up to 2GB but if I changed this setting in any way it would not Index the files. When I click to start indexing nothing would happen, would allow me to go back though. But if I left everything at the default settings it seemed to index everything just fine.
The reason for the large file size is that they are syslog files pulled from a Kiwi Syslog server, it stores each days logs in a basic text file.

Files Indexed 2
Emails Indexed 0
Errors 0
Warenings 0
Total Bytes 660.1MB
Unique Words 1306614

[David (PassMark)]

We'll check if there is a bug when setting very large file sizes.

If you are going to be indexing files where each file is 500MB+ in size, you are going to need a lot of RAM in your PC. It might be that you are just running out of RAM. The indexer is multithreaded. Meaning it will be allocating space in RAM to process several files of this size. So it is easy to run out of RAM if you are processing several files of this size at the same time.

A better approach is not to index any file over about 200MB. The log should then list what files were not indexed. You can then examine these files by hand to see what they are and if they warrant further attention.

It doesn't make sense to index just 1 large file. If you have just 1 large file, then you are better off just loading it up into the viewer in OSF or a normal text editor (e.g we like Ultra Edit) and doing your searching in the editor.

[Ray (PassMark)]

One of the files was over 600MB, I did try adjusting the settings to allow for a larger size file, it allowed me to set it up to 2GB but if I changed this setting in any way it would not Index the files. When I click to start indexing nothing would happen, would allow me to go back though.

We've confirmed this is a bug in the current release when Custom Limits are set in the Advanced window. It has been fixed for the next OSF release.