Implement a full SSL/TLS library

Got an idea for enhancing PureBasic? New command(s) you'd like to see?
User avatar
idle
Always Here
Always Here
Posts: 5019
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Implement a full SSL/TLS library

Post by idle »

plouf wrote: Wed Mar 02, 2022 8:38 pm if ever implemend i would also prefer to be external or as option external

the reason is that ssl changes so fast, and just replacing external libs, is easier for a number of ways, including user itself
yes I agree both would be ideal, you tend to get the static lib along with the shared object anyway.
User avatar
RichAlgeni
Addict
Addict
Posts: 914
Joined: Wed Sep 22, 2010 1:50 am
Location: Bradenton, FL

Re: Implement a full SSL/TLS library

Post by RichAlgeni »

Here are my thoughts: If I needed a Web Server, I could utilize one of a number available now. For instance, everything I personally do right now is on Windows machines, so I use Internet Information Server (IIS) 10. Like Apache, NGINX, and most other popular web servers, IIS has its own certificate management system, and they aren't too difficult to use. Since they are very to easy to use, to me it makes no sense to reinvent the wheel, so I would want the SSL/TLS library for a web server.

What I would be interested in, is a way to host a secure server communication process, that I could write to send secure messages back and forth to clients. I would like the ability to tell the secure server to use a particular certificate, so this could be a parameter when creating the server. Maybe something along the lines of this:

Code: Select all

secureServer.l = CreateSecureNetworkServer(#PB_Any, networkPort.l, #PB_Network_TCP|#PB_Network_IPv4, boundIP.s, certificatePath.s)
For the client, I'd like a method of indicating to the client it should ignore unauthenticated client and/or server certificates if needed. This could be done with additional flags added to the Mode parameter. Or in a way similar how CURL works, with #CURLOPT_SSL_VERIFYPEER and #CURLOPT_SSL_VERIFYHOST. If the traffic is important enough, both could be validated, or the socket would drop with an appropriate error.

It would be very nice if the server and client processes could renegotiate connection parameters, if necessary, in the background, and only alert with an error when a socket is, or must be dropped. For me, I have remote computers that need information pushed to them. Currently, I use Dynamic DNS to validate to server IP. This helps to make sure the host is valid. I have a list of valid client IP's, but it changes often, as a number of smaller clients still uses dynamic IP addresses. This works ok, but the data passed is clear text, which is not an ideal situation. I've gotten around this by encrypting the traffic before it is sent. If either the client, or the host do not recognize this encrypted traffic, the socket is 'hung out to dry,' so to speak. Data sent on the socket is read and discarded. Nothing is sent back. I've found this is the best way to handle potential intruders. However, it sure would be nice to just drop the data on the wire, using the normal SendNetworkData() command, without having to utilize pre-encrypt and post-decrypt procedures.
User avatar
idle
Always Here
Always Here
Posts: 5019
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Implement a full SSL/TLS library

Post by idle »

Thanks RichAlgeni, ideally it will just follow the established network code so it will only require minor changes to implement a TLS channel and also expose the lower level api so you can do what is required of it.

There is already a PB https webserver using libreSSL on the forum. He's using sockets natively which may be the result of a port from a c example or maybe it's due to an issue with PB network library. I'm going to look into that next so it will be available for windows and linux and it should help me work out how to wrap the library beyond just doing an adhock client server. I did find a couple of examples of client / server codes but I will have to dig around a bit more. For now I have enough to go on I think.

At the moment it will likely only be a shared object on windows with an import lib, I'm currently using mingw64 build and have yet to relink the static lib with dependencies to see if it works or leads to the same crt issues.
User avatar
RichAlgeni
Addict
Addict
Posts: 914
Joined: Wed Sep 22, 2010 1:50 am
Location: Bradenton, FL

Re: Implement a full SSL/TLS library

Post by RichAlgeni »

I'll take a look at that also. Thanks Idle!
ehowington
Enthusiast
Enthusiast
Posts: 114
Joined: Sat Sep 12, 2009 3:06 pm

Re: Implement a full SSL/TLS library

Post by ehowington »

+1
User avatar
Keya
Addict
Addict
Posts: 1891
Joined: Thu Jun 04, 2015 7:10 am

Re: Implement a full SSL/TLS library

Post by Keya »

It's 2022, we need the ability to easily talk to web servers that are secured by HTTPS ... which is pretty much all of them. In many ways this is probably the biggest NECESSITY for programmers of any language in this modern networked age - access to transparently use network protocols without having to worry too much about the underlying cryptography, as the crypto library should be doing that job. Even very simple apps still require HTTPS access. WE NEED EASY HTTPS ACCESS, we've needed it for years, and it's about time it just became a *normal* part of Purebasic, just as it's a seamless and normal part in other modern languages :) Thankyou
hoerbie
Enthusiast
Enthusiast
Posts: 119
Joined: Fri Dec 06, 2013 11:57 am
Location: DE/BY/MUC

Re: Implement a full SSL/TLS library

Post by hoerbie »

I'm 100% with Keya!
And I repeat my offer to pay extra for this, when it is integrated soon enough, before I move away from PB.
User avatar
captain_skank
Enthusiast
Enthusiast
Posts: 636
Joined: Fri Oct 06, 2006 3:57 pm
Location: England

Re: Implement a full SSL/TLS library

Post by captain_skank »

+1 from me and i would also be happy to pay for it.

Would be great to have it available for MySQL/MariDB and Postgres as well.
Fred
Administrator
Administrator
Posts: 16581
Joined: Fri May 17, 2002 4:39 pm
Location: France
Contact:

Re: Implement a full SSL/TLS library

Post by Fred »

Keya wrote: Sun May 08, 2022 7:00 am It's 2022, we need the ability to easily talk to web servers that are secured by HTTPS ... which is pretty much all of them. In many ways this is probably the biggest NECESSITY for programmers of any language in this modern networked age - access to transparently use network protocols without having to worry too much about the underlying cryptography, as the crypto library should be doing that job. Even very simple apps still require HTTPS access. WE NEED EASY HTTPS ACCESS, we've needed it for years, and it's about time it just became a *normal* part of Purebasic, just as it's a seamless and normal part in other modern languages :) Thankyou
You can already communicate with the HTTPS server with the HTTP lib, do you want to be able to open a raw connection on an https server ? Do you have any usecases in mind ? About creating your own HTTPS socket, I agree it can be useful and I will look into it.
hoerbie
Enthusiast
Enthusiast
Posts: 119
Joined: Fri Dec 06, 2013 11:57 am
Location: DE/BY/MUC

Re: Implement a full SSL/TLS library

Post by hoerbie »

@Fred: thanks for your reply, let's really talk about a full SSL/TLS library.

PB has at first sight some interesting network functions, but for example:
  • CreateNetworkServer or FTP don't have any security
  • SendMail through different hosting providers mail servers doesn't work secure, I think because of differences in StartTLS and full TLS
  • HTTPRequest* doesn't have an adjustable timeout
  • HTTPRequest* in Async mode around PB event handling isn't stable (see Windows bugs section, but this seems a general problem and not of SSL/TLS, or a wrong understanding of the help)
A lot of hosting providers will switch off unsecured connections in the future, so FTP and Mail will stop working someday, other providers switch off FTP and only allow SFTP.

Opening a secured server for access of others tools would be a big feature.
And there is (sorry to say that) also the any attention for to long missing SpiderBasic, that in the nearer future will get problems loading data from an unsecured PB server, when the browser programmers and mobile device industry goes further with security requirements.
Especially the combination of writing a PB server and SB app (Android/iOS) for me was promising some time ago.

Also from my point of view the Mail lib should be expanded, to that one can send a plain and html mail with included graphics and attachments. As this would be some work to implement, a simple "raw" mail function would help fast, so that anyone who knows how to put a good looking mail of different header lines and multipart body together can do it.

The lot of work for PB 6.0 with C compiler is really a big and best investment in the future of PB, but it doesn't help if PB looses requirements of the year 2022.

Just as a reminder: there was some discussion years ago about building a "feature request - vote - make a bid" system, maybe this could help you find out what is wanted mostly and to earn some additional money from integrating it. I think the game programmers will have other priorities as business software programmers like me.

Greets, hoerbie
plouf
Enthusiast
Enthusiast
Posts: 250
Joined: Fri Apr 25, 2003 6:35 pm
Location: Athens,Greece

Re: Implement a full SSL/TLS library

Post by plouf »

I suggest the "easy" way, already used by competive languanges

Is to use the curl library "layer" and/or just a curl library
As already suggest the implementation would be prefered to allow external link dll (sqlite way)
Because ssl change so fast and use a updated (or no) curl lib would allow always uptodate app without need to recompile each time
Christos
User avatar
skywalk
Addict
Addict
Posts: 3960
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Implement a full SSL/TLS library

Post by skywalk »

My interest is to communicate with my android or iOS phone.
There is talk of passwords going away and authentication via secure, verified mobile phone.
No way will Google and Apple allow http only transfers. Even if I scramble the data.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
idle
Always Here
Always Here
Posts: 5019
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Implement a full SSL/TLS library

Post by idle »

Fred wrote: Mon May 09, 2022 2:34 pm
Keya wrote: Sun May 08, 2022 7:00 am It's 2022, we need the ability to easily talk to web servers that are secured by HTTPS ... which is pretty much all of them. In many ways this is probably the biggest NECESSITY for programmers of any language in this modern networked age - access to transparently use network protocols without having to worry too much about the underlying cryptography, as the crypto library should be doing that job. Even very simple apps still require HTTPS access. WE NEED EASY HTTPS ACCESS, we've needed it for years, and it's about time it just became a *normal* part of Purebasic, just as it's a seamless and normal part in other modern languages :) Thankyou
You can already communicate with the HTTPS server with the HTTP lib, do you want to be able to open a raw connection on an https server ? Do you have any usecases in mind ? About creating your own HTTPS socket, I agree it can be useful and I will look into it.
Fred take a look at libressl it's easy to build and supports all platforms

https://www.libressl.org/releases.html

https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/

https://ftp.openbsd.org/pub/OpenBSD/Lib ... 5.2.tar.gz

Code: Select all

;* $OpenBSD: tls.h,v 1.58 2020/01/22 06:44:02 beck Exp $ */
;*
;* Copyright (c) 2014 Joel Sing <jsing@openbsd.org>
;*
;* Permission To use, copy, modify, And distribute this software For any
;* purpose With Or without fee is hereby granted, provided that the above
;* copyright notice And this permission notice appear in all copies.
;*
;* THE SOFTWARE IS PROVIDED "AS IS" And THE AUTHOR DISCLAIMS ALL WARRANTIES
;* With REGARD To THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;* MERCHANTABILITY And FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE For
;* ANY SPECIAL, DIRECT, INDIRECT, Or CONSEQUENTIAL DAMAGES Or ANY DAMAGES
;* WHATSOEVER RESULTING FROM LOSS OF USE, Data Or PROFITS, WHETHER IN AN
;* ACTION OF CONTRACT, NEGLIGENCE Or OTHER TORTIOUS ACTION, ARISING OUT OF
;* Or IN CONNECTION With THE USE Or PERFORMANCE OF THIS SOFTWARE.
;*


#TLS_API=	20200120

#TLS_PROTOCOL_TLSv1_0=	(1 << 1)
#TLS_PROTOCOL_TLSv1_1=	(1 << 2)
#TLS_PROTOCOL_TLSv1_2=	(1 << 3)
#TLS_PROTOCOL_TLSv1_3=	(1 << 4)

#TLS_PROTOCOL_TLSv1= #TLS_PROTOCOL_TLSv1_0 | #TLS_PROTOCOL_TLSv1_1 | #TLS_PROTOCOL_TLSv1_2 | #TLS_PROTOCOL_TLSv1_3

#TLS_PROTOCOLS_ALL= #TLS_PROTOCOL_TLSv1
#TLS_PROTOCOLS_DEFAULT= (#TLS_PROTOCOL_TLSv1_2|#TLS_PROTOCOL_TLSv1_3)

#TLS_WANT_POLLIN=	-2
#TLS_WANT_POLLOUT=	-3

;/* RFC 6960 Section 2.3 */
#TLS_OCSP_RESPONSE_SUCCESSFUL=		0
#TLS_OCSP_RESPONSE_MALFORMED=		1
#TLS_OCSP_RESPONSE_INTERNALERROR=		2
#TLS_OCSP_RESPONSE_TRYLATER=		3
#TLS_OCSP_RESPONSE_SIGREQUIRED=		4
#TLS_OCSP_RESPONSE_UNAUTHORIZED=		5

;/* RFC 6960 Section 2.2 */
#TLS_OCSP_CERT_GOOD=			0
#TLS_OCSP_CERT_REVOKED=			1
#TLS_OCSP_CERT_UNKNOWN=			2

;/* RFC 5280 Section 5.3.1 */
#TLS_CRL_REASON_UNSPECIFIED=		0
#TLS_CRL_REASON_KEY_COMPROMISE=		1
#TLS_CRL_REASON_CA_COMPROMISE=		2
#TLS_CRL_REASON_AFFILIATION_CHANGED=	3
#TLS_CRL_REASON_SUPERSEDED=		4
#TLS_CRL_REASON_CESSATION_OF_OPERATION=	5
#TLS_CRL_REASON_CERTIFICATE_HOLD=		6
#TLS_CRL_REASON_REMOVE_FROM_CRL=		8
#TLS_CRL_REASON_PRIVILEGE_WITHDRAWN=	9
#TLS_CRL_REASON_AA_COMPROMISE=		10

#TLS_MAX_SESSION_ID_LENGTH=		32
#TLS_TICKET_KEY_SIZE=			48

;download build from bsd  
;https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0.tar.gz

;Build tools required for windows builds cmake and mingw64
;cmake 
;https://cmake.org/download/ 

;mingw64
;Info https://www.mingw-w64.org/downloads/#llvm-mingw
;Available versions info https://github.com/mstorsjo/llvm-mingw/releases
;Direct download link https://github.com/mstorsjo/llvm-mingw/releases/download/20211002/llvm-mingw-20211002-msvcrt-x86_64.zip

;building on windows linux osx 
;copy the tar file to a folder of your choice, make a subfolder and then extract the tar and cd into the folder
;> mkdir libressl-3.5.0
;> tar -xf libressl-3.5.0.tar.gz
;> cd libressl-3.5.0

;Edit the file libressl-3.5.0\cmakelists and insert option(BUILD_SHARED_LIBS "Build shared" ON) at line 38  
;option(BUILD_SHARED_LIBS "Build shared" ON)

;now make a build folder cd to it and call cmake 
;libressl-3.5.0> mkdir build
;libressl-3.5.0> cd build 

;If OS = Windows
;  libressl-3.5.0\build> cmake -G="MinGW Makefiles"
;  libressl-3.5.0\build> mingw32-make all
;Else 
;  libressl-3.5.0\build> cmake all
;  libressl-3.5.0\build> make all
;EndIf 

PrototypeC tls_read_cb(ctx,*buf,_buflen,*cb_arg);
PrototypeC tls_write_cb(ctx,*buf,_buflen,*cb_arg);

CompilerIf #PB_Compiler_OS = #PB_OS_Windows 
    #LIBRESSLPATH = "libtls-24.lib"
CompilerElse 
    #LIBRESSLPATH = "libtls.a" 
CompilerEndIf       
  
ImportC #LIBRESSLPATH 
  tls_init()                                                                                                   
  tls_config_error(config)                                                                            
  tls_error(ctx)                                                                                      
  tls_config_new()                                                                                             
  tls_config_free(config)                                                                             
  tls_default_ca_cert_file()                                                                                   
  tls_config_add_keypair_file(config,cert_file.p-utf8,key_file.p-utf8)                              
  tls_config_add_keypair_mem(config,cert.p-utf8,certlen.i,key.p-utf8,key_len.i)        
  tls_config_add_keypair_ocsp_file(config,cert_file.p-utf8,key_file.p-utf8,ocsp_staple_file.p-utf8)
  tls_config_add_keypair_ocsp_mem(config,cert.p-utf8,cert_len.i,key.p-utf8,key_len.i,staple.p-utf8,staple_len.i)
  tls_config_set_alpn(config,alpn.p-utf8)                                                                                           
  tls_config_set_ca_file(config,ca_file.p-utf8)                                                                                     
  tls_config_set_ca_path(config,ca_path.p-utf8)                                                                                     
  tls_config_set_ca_mem(config,*ca,len.i)                                                                               
  tls_config_set_cert_file(config,cert_file.p-utf8)                                                                                 
  tls_config_set_cert_mem(config,*cert,len.i)                                                                           
  tls_config_set_ciphers(config,ciphers.p-utf8)                                                                                     
  tls_config_set_crl_file(config,crl_file.p-utf8)                                                                                   
  tls_config_set_crl_mem(config,crl.p-utf8,len.i)                                                                             
  tls_config_set_dheparams(config,params.p-utf8)                                                                                    
  tls_config_set_ecdhecurve(config,curve.p-utf8)                                                                                    
  tls_config_set_ecdhecurves(config,curves.p-utf8)                                                                                  
  tls_config_set_key_file(config,key_file.p-utf8)                                                                                   
  tls_config_set_key_mem(config,*key,len.i)                                                                             
  tls_config_set_keypair_file(config,cert_file.p-utf8,key_file.p-utf8)                                                             
  tls_config_set_keypair_mem(config,cert.p-utf8,cert_len.i,key.p-utf8,key_len.i)                                       
  tls_config_set_keypair_ocsp_file(config,cert_file.p-utf8,key_file.p-utf8,staple_file.p-utf8)                                    
  tls_config_set_keypair_ocsp_mem(config,cert.p-utf8,cert_len.i,key.p-utf8,key_len.i,staple.p-utf8,staple_len.i) 
  tls_config_set_ocsp_staple_mem(config,staple.p-utf8,len.i)                                                                  
  tls_config_set_ocsp_staple_file(config,staple_file.p-utf8)                                                                        
  tls_config_set_protocols(config,protocols.l)                                                                               
  tls_config_set_session_fd(config,session_fd.l)                                                                                  
  tls_config_set_verify_depth(config,verify_depth.l)                                                                              
  tls_config_prefer_ciphers_client(config)                                                                                           
  tls_config_prefer_ciphers_server(config)                                                                                           
  tls_config_insecure_noverifycert(config)                                                                                           
  tls_config_insecure_noverifyname(config)                                                                                           
  tls_config_insecure_noverifytime(config)                                                                                           
  tls_config_verify(config)                                                                                                          
  tls_config_ocsp_require_stapling(config)                                                                                           
  tls_config_verify_client(config)                                                                                                   
  tls_config_verify_client_optional(config)                                                                                          
  tls_config_clear_keys(config)                                                                                                      
  tls_config_parse_protocols(*protocols,protostr.p-utf8)                                                                           
  tls_config_set_session_id(config,session_id.p-utf8,len.i)                                                                   
  tls_config_set_session_lifetime(config,lifetime.l)                                                                              
  tls_config_add_ticket_key(config,keyrev.i,*key,keylen.i)                                                   
  tls_client()                                                                                                                                
  tls_server()                                                                                                                                
  tls_configure(ctx,config)                                                                                                 
  tls_reset(ctx)                                                                                                                     
  tls_free(ctx)                                                                                                                      
  tls_accept_fds(ctx,*cctx,fd_read.l,fd_write.l)                                                                       
  tls_accept_socket(ctx,*cctx,socket.l)                                                                                   
  tls_accept_cbs(ctx,*cctx,*read_cb.tls_read_cb,*write_cb.tls_write_cb,*cb_arg)                                      
  tls_connect(ctx,host.p-utf8,port.p-utf8)                                                                                         
  tls_connect_fds(ctx,fd_read.l,fd_write.l,servername.p-utf8)                                                                 
  tls_connect_servername(ctx,host.p-utf8,port.p-utf8,servername.p-utf8)                                                           
  tls_connect_socket(ctx,s.l,servername.p-utf8)                                                                                  
  tls_connect_cbs(ctx,*read_cb.tls_read_cb,*write_cb.tls_write_cb,*cb_arg,servername.p-utf8)                                
  tls_handshake(ctx)                                                                                                                 
  tls_read(ctx,*buf,buflen.i)                                                                                            
  tls_write(ctx,*buf,buflen.i)                                                                                          
  tls_close(ctx)                                                                                                                     
  tls_peer_cert_provided(ctx)                                                                                                        
  tls_peer_cert_contains_name(ctx,name.p-utf8)                                                                                      
  tls_peer_cert_hash(ctx)                                                                                                            
  tls_peer_cert_issuer(ctx)                                                                                                          
  tls_peer_cert_subject(ctx)                                                                                                         
  tls_peer_cert_notbefore(ctx)                                                                                                       
  tls_peer_cert_notafter(ctx)                                                                                                        
  tls_peer_cert_chain_pem(ctx,*len)                                                                                          
  tls_conn_alpn_selected(ctx)                                                                                                        
  tls_conn_cipher(ctx)                                                                                                               
  tls_conn_cipher_strength(ctx)                                                                                                      
  tls_conn_servername(ctx)                                                                                                           
  tls_conn_session_resumed(ctx)                                                                                                      
  tls_conn_version(ctx)                                                                                                              
  tls_load_file(file.p-utf8,*len,*password)                                                                                    
  tls_unload_file(*buf,len.i)                                                                                                   
  tls_ocsp_process_response(ctx,response.p-utf8,size.i)                                                                       
  tls_peer_ocsp_cert_status(ctx)                                                                                                     
  tls_peer_ocsp_crl_reason(ctx)                                                                                                      
  tls_peer_ocsp_next_update(ctx)                                                                                                     
  tls_peer_ocsp_response_status(ctx)                                                                                                 
  tls_peer_ocsp_result(ctx)                                                                                                          
  tls_peer_ocsp_revocation_time(ctx)                                                                                                 
  tls_peer_ocsp_this_update(ctx)                                                                                                     
  tls_peer_ocsp_url(ctx)                                                                                                             
EndImport


infratec
Always Here
Always Here
Posts: 6810
Joined: Sun Sep 07, 2008 12:45 pm
Location: Germany

Re: Implement a full SSL/TLS library

Post by infratec »

I found an old topic where I replied :oops:

https://www.purebasic.fr/english/viewto ... 70#p561670

Ground0 implemented a webserver with tls and libressl.

But unfortunately it is not eays to build a static libressl lib which is working with PB.
User avatar
idle
Always Here
Always Here
Posts: 5019
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: Implement a full SSL/TLS library

Post by idle »

infratec wrote: Tue May 10, 2022 7:30 am I found an old topic where I replied :oops:

https://www.purebasic.fr/english/viewto ... 70#p561670

Ground0 implemented a webserver with tls and libressl.

But unfortunately it is not eays to build a static libressl lib which is working with PB.
Yes I ported it to work on windows. I will share link later.
https://www.dropbox.com/s/vfkob35073v6fwv/lweb.zip?dl=0
Post Reply