README
¶
Fixed width file parser (decoder) for GO (golang)
Updated library derived from Oleg Lobanov's fwencoder with some aspects of Ian Lopshire's go-fixedwidth
Ths version currently only decodes but has a few additional features.
- It supports the TextMarshaler/TextUnmarshaler interface
- It allows multiple calls to the decoder by allowing a pointer to a struct to be passed to it as well as a slice.
- It's slightly faster because it caches conversion functions
- It supports arbitrary record endings and field conversions
- It allows the headers to be predefined by the caller
- It does not support JSON decoding for complex data structures.
- Encoding is also unsupported.
This library is using to parse fixed-width table data like:
Name Address Postcode Phone Credit Limit Birthday
Evan Whitehouse V4560 Camel Back Road 3122 (918) 605-5383 1000000.5 19870101
Chuck Norris P.O. Box 872 77868 (713) 868-6003 10909300 19651203
Install
To install the library use the following command:
$ go get -u github.com/goslogan/fw
Simple example
Parsing data from io.Reader:
type Person struct {
Name string
Address string
Postcode int
Phone string
CreditLimit float64 `json:"Credit Limit"`
Bday time.Time `column:"Birthday" format:"20060102"`
}
input, _ := os.ReadFile("/path/to/file")
defer f.Close
var people []Person
err := fw.Unmarshal(input, &people)
Documentation
¶
Overview ¶
Package fw defines a model for converting fixed width input data into Redis structs.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrIncorrectInputValue = errors.New("value is not a pointer to slice of structs or a pointer to a struct")
ErrIncorrectInputValue represents wrong input param
Functions ¶
func Unmarshal ¶
Unmarshal decodes a buffer into the array or structed pointed to by v If v is not an array only the first record will be read
Example ¶
source := []byte("name dob \nPeter 2008-10-11\nNicki 1987-01-28")
type Person struct {
Name string `column:"name"`
DOB time.Time `column:"dob" format:"2006-01-02"`
}
people := []Person{}
err := Unmarshal(source, &people)
if err != nil {
panic(err)
}
fmt.Printf("Number of records: %d\n", len(people))
for _, person := range people {
fmt.Printf("%s=%s\n", person.Name, person.DOB.Format(time.RFC822))
}
Output: Number of records: 2 Peter=11 Oct 08 00:00 UTC Nicki=28 Jan 87 00:00 UTC
func UnmarshalReader ¶
UnmarshalReader decodes an io.Reader into the array or structed pointed to by v If v is not an array only the first record will be read
Types ¶
type Decoder ¶
type Decoder struct {
RecordTerminator []byte // RecordTerminator identifies the sequence of bytes used to indicate end of record (default is "\n")
FieldSeparator string // FieldSeparator is used to identify the characters between fields and also to trim those characters. It's used as part of a regular expression (default is a space)
SkipFirstRecord bool // SkipFirstRecord defines whether the first line should be ignored.
// contains filtered or unexported fields
}
A Decoder reads and decodes fixed width data from an input stream. The caller can either define field sizes directly via Decoder.SetHeaders or they can be read from the first line of input.
Annotations ¶
Structs are annotated with the name of the input field/column with the column annotation. Referencing a column which does not exist will cause the field to be silently ignored during processing. Given the range of date/time formats in data, time.Time fields are supported additionally by the format annotation which allows the template for time.ParseDate to be provided.
Usable target structures ¶
The data structure passed to Decoder.Decode or Unmarshal must be a pointer to an existing slice or a pointer to a struct. If a slice is provided, it must contain structs or pointers to structs. It can be empty. Data is appended to the slice.
All basic go data types are supported automatically. As mentioned above time.Time is supported explicitly. Any other data type must support the encoding.TextUnmarshaler interface. Any other data type will cause an error to be returned.
Example ¶
source := []byte("name dob \nPeter 2008-10-11\nNicki 1987-01-28")
type Person struct {
Name string `column:"name"`
DOB time.Time `column:"dob" format:"2006-01-02"`
}
person := Person{}
decoder := NewDecoder(bytes.NewBuffer(source))
err := decoder.Decode(&person)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", person)
Output: {Name:Peter DOB:2008-10-11 00:00:00 +0000 UTC}
Example (Explicit) ¶
source := []byte("Peter 2008-10-11\nNicki 1987-01-28")
type Person struct {
Name string `column:"name"`
DOB time.Time `column:"dob" format:"2006-01-02"`
}
columns := map[string][]int{"name": {0, 8}, "dob": {8, 18}}
person := Person{}
decoder := NewDecoder(bytes.NewBuffer(source))
decoder.SetHeaders(columns)
err := decoder.Decode(&person)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", person)
Output: {Name:Peter DOB:2008-10-11 00:00:00 +0000 UTC}
func NewDecoder ¶
NewDecoder returns a new decoder that reads from r.
func (*Decoder) Decode ¶
Decode reads from its input and stores the decoded data to the value pointed to by v. v may point to a struct or a slice of structs (or pointers to structs)
Currently, the maximum decodable line length is bufio.MaxScanTokenSize-1. ErrTooLong is returned if a line is encountered that too long to decode.
func (*Decoder) SetHeaders ¶
SetHeaders overrides any headers parsed from the first line of input. If decoder.SetHeaders is called , decoder.SkipFirstRecord is set to false. If decoder.SkipFirstRecord is then set to true, the first line will be read but not parsed
type InvalidUnmarshalError ¶
An InvalidUnmarshalError describes an invalid argument passed to Unmarshal. (The argument to Unmarshal must be a non-nil pointer.)
func (*InvalidUnmarshalError) Error ¶
func (err *InvalidUnmarshalError) Error() string
Source Files