Harbour File Size Limits

 

Harbour File Size Limits

  • Max record size: 2^16-1 = 65535 byts ( 64 MB )
  • Max number of recors : 2^32-1 = 4,294,967,295 ( 4 Bilion )
  • Max .dbf file size : 2^48 = 256 TB
  • Max DBT memo file size : 2 TB
  • Max FPT memo file size : 256 GB
  • Max SMT memo file size : 128 GB
  • Max NTX file size (standard) : 4GB
  • Max NTX file size (incresead ) : 4TB
  • Max CDX file size : 4GB

 

Source : Harbour\doc\xhb-diff.txt :

### NATIVE RDDs ###

In both compilers maximal file size for tables, memos and indexes is limited only by OS and file format structures. Neither Harbour nor xHarbour introduce own limits here.

The maximal file size for DBFs is limited by number of records 2^32-1 = 4294967295 and maximal record size: 2^16-1 = 65535 what gives nearly 2^48 = 256TB as maximal .dbf file size.

The maximal memo format size depends on used memo type: DBT, FPT or SMT and size of  memo block. It’s limited by maximal number of memo blocks = 2^32 and size of memo block so it’s 2^32*<size_of_memo_block>.

The default memo block size for DBT is 512 bytes, FPT – 64 bytes and for SMT 32 bytes. So for standard memo block sizes the maximum are:

DBT->2TB, FPT->256GB, SMT->128GB. The maximal memo block size in Harbour is 2^32 and minimal is 1 byte and it can be any value between 1 and 65536 and then any number of 64KB blocks. The last limitation is introduced as workaround for some wrongly implemented in other
languages memo drivers which were setting only 16 bits in 32bit field in memo header. Most of other languages has limit for memo block size at 2^15 and the block size has to be power of 2. Some of them also introduce minimal block size limits. If programmers plans to share data with programs compiled by such languages then he should check their documentation to not create memo files which cannot be accessed by them.

Maximal NTX file size for standard NTX files is 4GB and it’s limited by internal NTX structures. Enabling 64bit locking in [x]Harbour change slightly used NTX format and increase maximum NTX file size to 4TB.

The NTX format in [x]Harbour has also many other extensions like support for multitag indexes or using record number as hidden part of index key and many others which are unique to [x]Harbour. In practice all of CDX extensions are supported by NTX in [x]Harbour.

The NSX format in [x]Harbour is also limited by default to 4GB but like in NTX enabling 64bit locking extend it to 4TB. It also supports common to NTX and CDX set of features.

The CDX format is limited to 4GB and so far [x]Harbour does not support extended mode which can increase the size up to 2TB with standard page length and it can be bigger in all formats if we introduce support for bigger index pages. Of course all such extended formats are not binary compatible with original ones and so far can be used only by [x]Harbour RDDs though in ADS the .adi format is such extended CDX format so maybe in the future it will be possible to use .adi indexes in our CDX RDD.

Of course all of the above sizes can be reduced by operating system (OS) or file system (FS) limitations so it’s necessary to check what is supported by environment where [x]Harbour applications are executed. 

Data Type Terms

Character :

A special data type consisting of one or more values in the extended character set. Characters can be grouped together to form strings. The maximum size of a character string In Clipper language is 65,534 bytes.

In Clipper, a character data may include any character, including “control” or “unprintable” characters, that are ASCII code less than 32.

Note : Characters with ASCII code 9, 10 and 13 always treated as “white spaces”; this important with EMPTY() and xTRIM() functions.

See Also:  String

Code Block :

A special data type that refers to a piece of compiled program code. In a program, the source code that specifies the creation of a code block.

Data Type :

The category of a data value. A data type is distinguished by the set of allowable values for that type, the set of operators that can be applied, and the storage format used to represent these values. In Clipper language, the following data types are defined: character, numeric, date, logical, array, object, code block, and NIL. Program variables may contain values of any type. Database field variables are limited to character, numeric, date, logical, and a special type called memo which is treated the same as character.

Date Type :

A special data type consisting of digits to store year, month, and day values. Operations on date values are based on chronological values.

Integer :

A number with no decimal digits. Note that Clipper language does not provide a separate data type for integer values.

Literal :

A source code element interpreted literally (as encountered), and assumed to have no abstract meaning. Generally a constant.

See Also: Constant

Logical Type :

A special data type consisting of true (.T.) or false (.F.) values.

See Also: Condition

Memo Type :

A special database field type consisting of one or more characters in the extended character set. The maximum size of a memo field in Clipper language is 65,534 bytes. A memo field differs only from a character string by the fact it is stored in a separate memo (.DBT file) and the field length is variable-length.

See Also: Character String

NIL :

A special data type that has only one allowable value. The special value (NIL) is automatically assigned to all uninitialized variables except publics, and is also passed as a substitute when arguments are omitted in a procedure or function call.

Numeric Type :

A special data type consisting of values that indicate magnitude. Numeric values consist of digits between zero and nine, a sign, and a decimal point.

Objects :

An object is an instance of a class. Each object has one or more attributes (called instance variables) and a series of operations (methods) that execute when a message is sent to the object. The object’s instance variables can only be accessed or assigned by sending messages to the object. Objects are created by calling a special function associated with a class.

See Also: Classes, Instance Variables, Messages, Methods

String :

Generically, a value of type character. In source code, a series of characters enclosed in single or double quotes.

See Also : Character

Substring :

A string within a string, usually to be specified as an argument of a function or command.

Database Terms

Alias :

The name of a work area; an alternate name given to a database file. Aliases are often used to give database files descriptive names and are assigned when the database file is opened. If no alias is specified when the database file is USEd, the name of the database file becomes the alias.

An alias can be used to reference both fields and expressions (including user-defined functions). In order to alias an expression, the expression must be enclosed in parentheses.

See also : Work Area

Attribute :

As a formal DBMS term, refers to a column or field in a table or database file.

See Also: Column, Field

Beginning of File :

The top of the database file. In Clipper language there is no beginning of file area or record. Instead, it is indicated by BOF() returning true (.T.) if an attempt is made to move the record pointer above the first record in the database file or the database file is empty.

Cell :

In a table, a cell is the intersection of a Row and a Column.

Column :

A database term used to describe a field in a table or database file.

See Also: Field

Concurrency :

The degree to which data can be accessed by more than one user at the same time.

Condition :

A logical expression that determines whether an operation will take place. With database commands, a logical expression that determines what records are included in an operation. Conditions are specified as arguments of the FOR or WHILE clause.

See Also: Scope

Controlling/Master Index :

The index currently being used to refer to records by key value or sequential record movement commands.

See Also: Index, Natural Order

Database :

An aggregation of related operational data used by an application system. A database can contain one or more data files or tables.

See Also: Field, Record, Tuple, View

DBMS :

An acronym for the term database management system. A DBMS is a software system that mediates access to a database through a data manipulation language.

Delimited File :

A text file that contains variable-length database records with each record separated by a carriage return/linefeed pair (CHR(13) + CHR(10)) and terminated with an end of file mark (CHR(26)). Each field within a delimited file is variable length, not padded with either leading or trailing spaces, and separated by a comma. Character strings are optionally delimited to allow for embedded commas.

End of File :

The bottom of a database file. In Clipper language, this is LASTREC() + 1 and is indicated by EOF() returning true (.T.).

Field :

The basic column unit of a database file. A field has four attributes: name, type, length, and decimals if the type is numeric.

See Also: Database, Record, Tuple, Vector, View

Field Variable :

A variable that refers to data in a database field, as opposed to data in memory.

See Also: Local Variable, Memory Variable, Variable

Index :

An ordered set of key values that provides a logical ordering of the records in an associated database file. Each key in an index is associated with a particular record in the database file. The records can be processed sequentially in key order, and any record can be located by performing a SEEK operation with the associated key value.

See Also: Controlling/Master Index, Key Value, Natural Order

Join :

An operation that takes two tables as operands and produces one table as a result. It is, in fact, a combination of other operations including selection and projection.

See Also: Projection, Selection

Key Expression :

An expression, typically based on one or more database fields, that when evaluated, yields a key value for a database record. Key expressions are most often used to create indexes or for summarization operations.

See Also: Index, Key Value

Key Value :

The value produced by evaluating a key expression. When placed in an index, a key value identifies the logical position of the associated record in its database file.

See Also: Index, Key Expression

Master Index :

The index currently being used to refer to records by key value or sequential record movement commands.

See Also : Controlling/Master Index

Memo Type :

A special database field type consisting of one or more characters in the extended character set. The maximum size of a memo field In Clipper language is 65,534 bytes. A memo field differs only from a character string by the fact it is stored in a separate memo (.DBT file) and the field length is variable-length.

See Also: Character String

Natural Order :

For a database file, the order determined by the sequence in which records were originally entered into the file. Also called unindexed order.

See Also: Index

Normalization :

The process of elimination and consolidation of redundant data elements in a database system.

Projection :

A DBMS term specifying a subset of fields. In Clipper, the analogy is the FIELDS clause.

See Also: Join Selection

Query :

A request for information to be retrieved from a database. Alternately, a data structure in which such a request is encoded.

Record :

The basic row unit of a database file consisting of one or more field elements.

See Also: Database, Field, Table, Tuple

Relation :

A link between database files that allows the record pointer to move in more than one database file based on the value of a common field or expression. This allows information to be accessed from more than one database file at a time.

Relational Database System :

A system that stores data in rows and columns, without system dependencies within the data. In other words, relationships between different databases are not stored in the actual database itself, as is the case in a system that uses record pointers.

Row :

A group of related column or field values that are treated as a single entity. It is the same as a Clipper language record.

See Also: Column, Field, Record

Search Condition :

See : Condition, Scope

Scope :

In a database command, a clause that specifies a range of database records to be addressed by the command. The scope clause uses the qualifiers ALL, NEXT, RECORD, and REST to define the record scope.

See Also: Condition

SDF File :

A text file that contains fixed-length database records with each record separated by a carriage return/linefeed pair (CHR(13) + CHR(10)) and terminated with an end of file mark (CHR(26)). Each field within an SDF file is fixed-length with character strings padded with trailing spaces and numeric values padded with leading spaces. There are no field separators.

See Also: Database, Delimited File, Text File

Selection :

A DBMS term that specifies a subset of records meeting a condition. The selection itself is obtained with a selection operator. In Clipper language, the analogy is the FOR clause.

Separator :

The character or set of characters that differentiate fields or records from one another. In Clipper language, the DELIMITED and SDF file types have separators. The DELIMITED file uses a comma as the field separator and a carriage return/linefeed pair as the record separator. The SDF file type has no field separator, but also uses a carriage return/linefeed pair as the record separator.

See Also: Delimiter

Sort Order :

Describes the various ways database files and arrays are ordered.

. Ascending

Causes the order of data in a sort to be from lowest value to highest value.

. Descending

Causes the order of data in a sort to be from highest value to lowest value.

. Chronological

Causes data in a sort to be ordered based on a date value, from earliest to most recent.

. ASCII

Causes data in a sort to be ordered according to the ASCII Code values of the data to be sorted.

. Dictionary

The data in a sort is ordered in the way it would appear if the items sorted were entries in a dictionary of the English language.

. Collating Sequence

Data in a sort will be placed in sequence following the order of characters in the Extended Character Set.

. Natural

The order in which data was entered into the database.

Table :

A DBMS term defining a collection of column definitions and row values. In Clipper, it is represented and referred to as a database file.

Tuple :

A formal DBMS term that refers to a row in a table or a record in a database file. In DIF files, tuple also refers to the equivalent of a table record.

See Also: Database, Field, Record

Update :

The process of changing the value of fields in one or more records. Database fields are updated by various commands and the assignment operator.

Vector :

In a DIF file, vector refers to the equivalent of a table field.

See Also: Database, Field, Record, Tuple

View :

A DBMS term that defines a virtual table. A virtual table does not actually exist but is derived from existing tables and maintained as a definition. The definition in turn is maintained in a separate file or as an entry in a system dictionary file. In Clipper, views are supported only by DBU.EXE and are maintained in (.vew) files.

See Also: Database, Field, Record

Work Area :

The basic containment area of a database file and its associated indexes. Work areas can be referred to by alias name, number, or a letter designator.

See Also: Alias