README
¶
httplib
httplib is an libs help you to curl remote url.
How to use?
GET
you can use Get to crawl data.
import "gitee.com/xingnan/toolbox/httplib"
str, err := httplib.Get("http://baidu.com/").String()
if err != nil {
// error
}
fmt.Println(str)
POST
POST data to remote url
req := httplib.Post("http://baidu.com/")
req.Param("username","baidusb")
req.Param("password","123456")
str, err := req.String()
if err != nil {
// error
}
fmt.Println(str)
Set timeout
The default timeout is 60 seconds, function prototype:
SetTimeout(connectTimeout, readWriteTimeout time.Duration)
Example:
// GET
httplib.Get("http://baidu.com").SetTimeout(100 * time.Second, 30 * time.Second)
// POST
httplib.Post("http://baidu.com").SetTimeout(100 * time.Second, 30 * time.Second)
Debug
If you want to debug the request info, set the debug on
httplib.Get("http://baidu.com").Debug(true)
Set HTTP Basic Auth
str, err := Get("http://baidu.com").SetBasicAuth("user", "passwd").String()
if err != nil {
// error
}
fmt.Println(str)
Set HTTPS
If request url is https, You can set the client support TSL:
httplib.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: true})
More info about the tls.Config please visit http://golang.org/pkg/crypto/tls/#Config
Set HTTP Version
some servers need to specify the protocol version of HTTP
httplib.Get("http://baidu.com").SetProtocolVersion("HTTP/1.1")
Set Cookie
some http request need setcookie. So set it like this:
cookie := &http.Cookie{}
cookie.Name = "username"
cookie.Value = "baidusb"
httplib.Get("http://baidu.com").SetCookie(cookie)
Upload file
httplib support mutil file upload, use req.PostFile()
req := httplib.Post("baidu.com")
req.Param("username","baidusb")
req.PostFile("uploadfile1", "httplib.pdf")
str, err := req.String()
if err != nil {
// error
}
fmt.Println(str)
Documentation
¶
Index ¶
- func SetDefaultSetting(setting ToolboxHTTPSettings)
- func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)
- type ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) Body(data interface{}) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) Bytes() ([]byte, error)
- func (b *ToolboxHTTPRequest) Debug(isdebug bool) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) DoRequest() (resp *http.Response, err error)
- func (b *ToolboxHTTPRequest) DumpBody(isdump bool) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) DumpRequest() []byte
- func (b *ToolboxHTTPRequest) GetRequest() *http.Request
- func (b *ToolboxHTTPRequest) Header(key, value string) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) JSONBody(obj interface{}) (*ToolboxHTTPRequest, error)
- func (b *ToolboxHTTPRequest) Param(key, value string) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) PostFile(formname, filename string) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) Response() (*http.Response, error)
- func (b *ToolboxHTTPRequest) Retries(times int) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetBasicAuth(username, password string) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetCheckRedirect(redirect func(req *http.Request, via []*http.Request) error) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetCookie(cookie *http.Cookie) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetEnableCookie(enable bool) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetHost(host string) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetProtocolVersion(vers string) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetTLSClientConfig(config *tls.Config) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetTimeout(connectTimeout, readWriteTimeout time.Duration) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetTransport(transport http.RoundTripper) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) SetUserAgent(useragent string) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) Setting(setting ToolboxHTTPSettings) *ToolboxHTTPRequest
- func (b *ToolboxHTTPRequest) String() (string, error)
- func (b *ToolboxHTTPRequest) ToFile(filename string) error
- func (b *ToolboxHTTPRequest) ToJSON(v interface{}) error
- func (b *ToolboxHTTPRequest) ToXML(v interface{}) error
- func (b *ToolboxHTTPRequest) XMLBody(obj interface{}) (*ToolboxHTTPRequest, error)
- type ToolboxHTTPSettings
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SetDefaultSetting ¶
func SetDefaultSetting(setting ToolboxHTTPSettings)
SetDefaultSetting Overwrite default settings
Types ¶
type ToolboxHTTPRequest ¶
type ToolboxHTTPRequest struct {
// contains filtered or unexported fields
}
ToolboxHTTPRequest provides more useful methods for requesting one url than http.Request.
func Delete ¶
func Delete(url string) *ToolboxHTTPRequest
Delete returns *ToolboxHTTPRequest DELETE method.
func Get ¶
func Get(url string) *ToolboxHTTPRequest
Get returns *ToolboxHTTPRequest with GET method.
func Head ¶
func Head(url string) *ToolboxHTTPRequest
Head returns *ToolboxHTTPRequest with HEAD method.
func NewHttpRequest ¶
func NewHttpRequest(rawurl, method string) *ToolboxHTTPRequest
NewToolboxRequest return *ToolboxHTTPRequest with specific method
func Post ¶
func Post(url string) *ToolboxHTTPRequest
Post returns *ToolboxHTTPRequest with POST method.
func Put ¶
func Put(url string) *ToolboxHTTPRequest
Put returns *ToolboxHTTPRequest with PUT method.
func (*ToolboxHTTPRequest) Body ¶
func (b *ToolboxHTTPRequest) Body(data interface{}) *ToolboxHTTPRequest
Body adds request raw body. it supports string and []byte.
func (*ToolboxHTTPRequest) Bytes ¶
func (b *ToolboxHTTPRequest) Bytes() ([]byte, error)
Bytes returns the body []byte in response. it calls Response inner.
func (*ToolboxHTTPRequest) Debug ¶
func (b *ToolboxHTTPRequest) Debug(isdebug bool) *ToolboxHTTPRequest
Debug sets show debug or not when executing request.
func (*ToolboxHTTPRequest) DoRequest ¶
func (b *ToolboxHTTPRequest) DoRequest() (resp *http.Response, err error)
DoRequest will do the client.Do
func (*ToolboxHTTPRequest) DumpBody ¶
func (b *ToolboxHTTPRequest) DumpBody(isdump bool) *ToolboxHTTPRequest
DumpBody setting whether need to Dump the Body.
func (*ToolboxHTTPRequest) DumpRequest ¶
func (b *ToolboxHTTPRequest) DumpRequest() []byte
DumpRequest return the DumpRequest
func (*ToolboxHTTPRequest) GetRequest ¶
func (b *ToolboxHTTPRequest) GetRequest() *http.Request
GetRequest return the request object
func (*ToolboxHTTPRequest) Header ¶
func (b *ToolboxHTTPRequest) Header(key, value string) *ToolboxHTTPRequest
Header add header item string in request.
func (*ToolboxHTTPRequest) JSONBody ¶
func (b *ToolboxHTTPRequest) JSONBody(obj interface{}) (*ToolboxHTTPRequest, error)
JSONBody adds request raw body encoding by JSON.
func (*ToolboxHTTPRequest) Param ¶
func (b *ToolboxHTTPRequest) Param(key, value string) *ToolboxHTTPRequest
Param adds query param in to request. params build query string as ?key1=value1&key2=value2...
func (*ToolboxHTTPRequest) PostFile ¶
func (b *ToolboxHTTPRequest) PostFile(formname, filename string) *ToolboxHTTPRequest
PostFile add a post file to the request
func (*ToolboxHTTPRequest) Response ¶
func (b *ToolboxHTTPRequest) Response() (*http.Response, error)
Response executes request client gets response mannually.
func (*ToolboxHTTPRequest) Retries ¶
func (b *ToolboxHTTPRequest) Retries(times int) *ToolboxHTTPRequest
Retries sets Retries times. default is 0 means no retried. -1 means retried forever. others means retried times.
func (*ToolboxHTTPRequest) SetBasicAuth ¶
func (b *ToolboxHTTPRequest) SetBasicAuth(username, password string) *ToolboxHTTPRequest
SetBasicAuth sets the request's Authorization header to use HTTP Basic Authentication with the provided username and password.
func (*ToolboxHTTPRequest) SetCheckRedirect ¶
func (b *ToolboxHTTPRequest) SetCheckRedirect(redirect func(req *http.Request, via []*http.Request) error) *ToolboxHTTPRequest
SetCheckRedirect specifies the policy for handling redirects.
If CheckRedirect is nil, the Client uses its default policy, which is to stop after 10 consecutive requests.
func (*ToolboxHTTPRequest) SetCookie ¶
func (b *ToolboxHTTPRequest) SetCookie(cookie *http.Cookie) *ToolboxHTTPRequest
SetCookie add cookie into request.
func (*ToolboxHTTPRequest) SetEnableCookie ¶
func (b *ToolboxHTTPRequest) SetEnableCookie(enable bool) *ToolboxHTTPRequest
SetEnableCookie sets enable/disable cookiejar
func (*ToolboxHTTPRequest) SetHost ¶
func (b *ToolboxHTTPRequest) SetHost(host string) *ToolboxHTTPRequest
SetHost set the request host
func (*ToolboxHTTPRequest) SetProtocolVersion ¶
func (b *ToolboxHTTPRequest) SetProtocolVersion(vers string) *ToolboxHTTPRequest
SetProtocolVersion Set the protocol version for incoming requests. Client requests always use HTTP/1.1.
func (*ToolboxHTTPRequest) SetProxy ¶
func (b *ToolboxHTTPRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *ToolboxHTTPRequest
SetProxy set the http proxy example:
func(req *http.Request) (*url.URL, error) {
u, _ := url.ParseRequestURI("http://127.0.0.1:8118")
return u, nil
}
func (*ToolboxHTTPRequest) SetTLSClientConfig ¶
func (b *ToolboxHTTPRequest) SetTLSClientConfig(config *tls.Config) *ToolboxHTTPRequest
SetTLSClientConfig sets tls connection configurations if visiting https url.
func (*ToolboxHTTPRequest) SetTimeout ¶
func (b *ToolboxHTTPRequest) SetTimeout(connectTimeout, readWriteTimeout time.Duration) *ToolboxHTTPRequest
SetTimeout sets connect time out and read-write time out for Request.
func (*ToolboxHTTPRequest) SetTransport ¶
func (b *ToolboxHTTPRequest) SetTransport(transport http.RoundTripper) *ToolboxHTTPRequest
SetTransport set the setting transport
func (*ToolboxHTTPRequest) SetUserAgent ¶
func (b *ToolboxHTTPRequest) SetUserAgent(useragent string) *ToolboxHTTPRequest
SetUserAgent sets User-Agent header field
func (*ToolboxHTTPRequest) Setting ¶
func (b *ToolboxHTTPRequest) Setting(setting ToolboxHTTPSettings) *ToolboxHTTPRequest
Setting Change request settings
func (*ToolboxHTTPRequest) String ¶
func (b *ToolboxHTTPRequest) String() (string, error)
String returns the body string in response. it calls Response inner.
func (*ToolboxHTTPRequest) ToFile ¶
func (b *ToolboxHTTPRequest) ToFile(filename string) error
ToFile saves the body data in response to one file. it calls Response inner.
func (*ToolboxHTTPRequest) ToJSON ¶
func (b *ToolboxHTTPRequest) ToJSON(v interface{}) error
ToJSON returns the map that marshals from the body bytes as json in response . it calls Response inner.
func (*ToolboxHTTPRequest) ToXML ¶
func (b *ToolboxHTTPRequest) ToXML(v interface{}) error
ToXML returns the map that marshals from the body bytes as xml in response . it calls Response inner.
func (*ToolboxHTTPRequest) XMLBody ¶
func (b *ToolboxHTTPRequest) XMLBody(obj interface{}) (*ToolboxHTTPRequest, error)
XMLBody adds request raw body encoding by XML.
type ToolboxHTTPSettings ¶
type ToolboxHTTPSettings struct {
ShowDebug bool
UserAgent string
ConnectTimeout time.Duration
ReadWriteTimeout time.Duration
TLSClientConfig *tls.Config
Proxy func(*http.Request) (*url.URL, error)
Transport http.RoundTripper
CheckRedirect func(req *http.Request, via []*http.Request) error
EnableCookie bool
Gzip bool
DumpBody bool
Retries int // if set to -1 means will retry forever
}
ToolboxHTTPSettings is the http.Client setting
Source Files