Add master and cert log actions

Also, improve documentation

NEWS.md

@@ -15,7 +15,7 @@
          uri (uri[*]|*)|
          ip (serveraddr|*)|
          *)]
-      [log ([connect] [content] [pcap] [mirror]|*)]
+      [log ([connect] [master] [cert] [content] [pcap] [mirror]|*)]
       |*)
 
 -   Add -Q test config option.

README.md

@@ -280,15 +280,15 @@ only.
 SSLproxy can divert, split, pass, block, or match connections based on 
 filtering rules. Filtering rules can be defined globally or per-proxyspec.
 
-- Divert action diverts packets to listening program, allowing SSL inspection 
+- `Divert` action diverts packets to listening program, allowing SSL inspection 
 by listening program and content logging of packets
-- Split action splits the connection but does not divert packets to listening 
+- `Split` action splits the connection but does not divert packets to listening 
 program, effectively disabling SSL inspection by listening program, but 
 allowing content logging of packets
-- Pass action passes the connection through by engaging passthrough mode, 
+- `Pass` action passes the connection through by engaging passthrough mode, 
 effectively disabling SSL inspection and content logging of packets
-- Block action terminates the connection
-- Match action specifies log actions for the connection without changing its 
+- `Block` action terminates the connection
+- `Match` action specifies log actions for the connection without changing its 
 filter action
 
 The syntax of filtering rules is as follows:
@@ -305,7 +305,7 @@ The syntax of filtering rules is as follows:
 	     uri (uri[*]|*)|
 	     ip (serveraddr|*)|
 	     *)]
-	  [log ([connect] [content] [pcap] [mirror]|*)]
+	  [log ([connect] [master] [cert] [content] [pcap] [mirror]|*)]
 	  |*)
 
 The definition of which connections the rule action will be applied to is 
@@ -322,10 +322,14 @@ Common Names of SSL connections, Host or URI fields in HTTP Request headers, or
 connection.
 
 If and how a connection should be logged is specified using the `log` part of 
-filtering rules. `connect` enables logging connection information to connect 
-log file, `content` enables logging packet contents to content log file, 
-`pcap` enables writing packets to pcap files, and `mirror` enables mirroring 
-packets to mirror interfaces or targets.
+filtering rules:
+
+- `connect` enables logging connection information to connect log file
+- `master` enables logging of master keys
+- `cert` enables logging of generated certificates
+- `content` enables logging packet contents to content log file
+- `pcap` enables writing packets to pcap files
+- `mirror` enables mirroring packets to mirror interfaces or targets
 
 For example, if the following rules are defined in a structured HTTPS proxyspec,
 
@@ -363,8 +367,8 @@ Filtering rules are applied based on certain precedence orders:
 - The precedence of filter types is as Dst Host > SSL > HTTP.
 - The precedence of filter actions is as Divert > Split > Pass > Block. This is 
 only for the same type of filter rules.
-- The precedence of site fields is as sni > cn for ssl filter and host > uri 
-for http filter.
+- The precedence of site fields is as sni > cn for SSL filter and host > uri 
+for HTTP filter.
 
 For example, the pass action of a Dst Host filter rule is taken before the 
 split action of an SSL filter rule with the same from definition, due to the 
@@ -374,9 +378,10 @@ to the precedence order of site fields.
 
 In terms of possible filter actions,
 
-- Dst Host filter rules can take all of the actions.
-- SSL filter rules can take all of the actions.
-- HTTP filter rules can take the block action, but not divert, split, or pass 
+- Dst Host filter rules can take all of the filter and log actions.
+- SSL filter rules can take all of the filter and log actions.
+- HTTP filter rules can take the match and block filter actions, but not 
+the divert, split, or pass actions. Also, HTTP filter rules cannot take log 
 actions.
 
 Log actions do not configure any loggers. Global loggers for respective log 
@@ -390,7 +395,8 @@ You can append an asterisk `*` to site field of filtering rules for substring
 matching. Otherwise, the filter searches for an exact match with the site field 
 in the rule.
 
-The order of `from`, `to`, and `log` parts is not important.
+The order of from, to, and log parts is not important. The order of log 
+actions is not important.
 
 If the UserAuth option is disabled, only client IP addresses can be used in 
 the from part of filtering rules.

src/opts.c

@@ -786,6 +786,8 @@ clone_global_opts(global_t *global, const char *argv0, tmp_global_opts_t *tmp_gl
 		fr->match = rule->match;
 
 		fr->log_connect = rule->log_connect;
+		fr->log_master = rule->log_master;
+		fr->log_cert = rule->log_cert;
 		fr->log_content = rule->log_content;
 		fr->log_pcap = rule->log_pcap;
 #ifndef WITHOUT_MIRROR
@@ -1137,7 +1139,7 @@ filter_rule_str(filter_rule_t *rule)
 #ifndef WITHOUT_USERAUTH
 				"|%s"
 #endif /* !WITHOUT_USERAUTH */
-				"|%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s"
+				"|%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s|%s|%s"
 #ifndef WITHOUT_MIRROR
 				"|%s"
 #endif /* !WITHOUT_MIRROR */
@@ -1152,7 +1154,7 @@ filter_rule_str(filter_rule_t *rule)
 #endif /* !WITHOUT_USERAUTH */
 				rule->all_sites ? "sites" : "",
 				rule->divert ? "divert" : "", rule->split ? "split" : "", rule->pass ? "pass" : "", rule->block ? "block" : "", rule->match ? "match" : "",
-				rule->log_connect ? "connect" : "", rule->log_content ? "content" : "", rule->log_pcap ? "pcap" : "",
+				rule->log_connect ? "connect" : "", rule->log_master ? "master" : "", rule->log_cert ? "cert" : "", rule->log_content ? "content" : "", rule->log_pcap ? "pcap" : "",
 #ifndef WITHOUT_MIRROR
 				rule->log_mirror ? "mirror" : "",
 #endif /* !WITHOUT_MIRROR */
@@ -1191,14 +1193,14 @@ filter_sites_str(filter_site_t *site)
 	int count = 0;
 	while (site) {
 		char *p;
-		if (asprintf(&p, "%s\n      %d: %s (%s%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s"
+		if (asprintf(&p, "%s\n      %d: %s (%s%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s|%s|%s"
 #ifndef WITHOUT_MIRROR
 				"|%s"
 #endif /* !WITHOUT_MIRROR */
 				")", STRORNONE(s), count,
 				site->site, site->all_sites ? "all_sites, " : "", site->exact ? "exact" : "substring",
 				site->divert ? "divert" : "", site->split ? "split" : "", site->pass ? "pass" : "", site->block ? "block" : "", site->match ? "match" : "",
-				site->log_connect ? "connect" : "", site->log_content ? "content" : "", site->log_pcap ? "pcap" : ""
+				site->log_connect ? "connect" : "", site->log_master ? "master" : "", site->log_cert ? "cert" : "", site->log_content ? "content" : "", site->log_pcap ? "pcap" : ""
 #ifndef WITHOUT_MIRROR
 				, site->log_mirror ? "mirror" : ""
 #endif /* !WITHOUT_MIRROR */
@@ -2434,7 +2436,7 @@ opts_set_passsite(opts_t *opts, char *value, int line_num)
 #ifndef WITHOUT_USERAUTH
 		"%s|"
 #endif /* !WITHOUT_USERAUTH */
-		"%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s"
+		"%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s|%s|%s"
 #ifndef WITHOUT_MIRROR
 		"|%s"
 #endif /* !WITHOUT_MIRROR */
@@ -2449,7 +2451,7 @@ opts_set_passsite(opts_t *opts, char *value, int line_num)
 #endif /* !WITHOUT_USERAUTH */
 		rule->all_sites ? "sites" : "",
 		rule->divert ? "divert" : "", rule->split ? "split" : "", rule->pass ? "pass" : "", rule->block ? "block" : "", rule->match ? "match" : "",
-		rule->log_connect ? "connect" : "", rule->log_content ? "content" : "", rule->log_pcap ? "pcap" : "",
+		rule->log_connect ? "connect" : "", rule->log_master ? "master" : "", rule->log_cert ? "cert" : "", rule->log_content ? "content" : "", rule->log_pcap ? "pcap" : "",
 #ifndef WITHOUT_MIRROR
 		rule->log_mirror ? "mirror" : "",
 #endif /* !WITHOUT_MIRROR */
@@ -2514,7 +2516,7 @@ filter_rule_parse(opts_t *opts, const char *name, char *value, int line_num)
 	//     uri (uri[*]|*)|
 	//     ip (serveraddr|*)|
 	//     *)]
-	//  [log ([connect] [content] [pcap] [mirror]|*)]
+	//  [log ([connect] [master] [cert] [content] [pcap] [mirror]|*)]
 	//  |*)
 
 	char *argv[sizeof(char *) * MAX_FILTER_RULE_TOKENS];
@@ -2664,7 +2666,7 @@ filter_rule_parse(opts_t *opts, const char *name, char *value, int line_num)
 			}
 
 			i = opts_inc_arg_index(i, argc, argv[i], line_num);
-			if (equal(argv[i], "connect") || equal(argv[i], "content") || equal(argv[i], "pcap")
+			if (equal(argv[i], "connect") || equal(argv[i], "master") || equal(argv[i], "cert") || equal(argv[i], "content") || equal(argv[i], "pcap")
 #ifndef WITHOUT_MIRROR
 				|| equal(argv[i], "mirror")
 #endif /* !WITHOUT_MIRROR */
@@ -2672,6 +2674,10 @@ filter_rule_parse(opts_t *opts, const char *name, char *value, int line_num)
 				do {
 					if (equal(argv[i], "connect"))
 						rule->log_connect = 1;
+					else if (equal(argv[i], "master"))
+						rule->log_master = 1;
+					else if (equal(argv[i], "cert"))
+						rule->log_cert = 1;
 					else if (equal(argv[i], "content"))
 						rule->log_content = 1;
 					else if (equal(argv[i], "pcap"))
@@ -2683,7 +2689,7 @@ filter_rule_parse(opts_t *opts, const char *name, char *value, int line_num)
 
 					if (++i == argc)
 						break;
-				} while (equal(argv[i], "connect") || equal(argv[i], "content") || equal(argv[i], "pcap")
+				} while (equal(argv[i], "connect") || equal(argv[i], "master") || equal(argv[i], "cert") || equal(argv[i], "content") || equal(argv[i], "pcap")
 #ifndef WITHOUT_MIRROR
 					|| equal(argv[i], "mirror")
 #endif /* !WITHOUT_MIRROR */
@@ -2693,6 +2699,8 @@ filter_rule_parse(opts_t *opts, const char *name, char *value, int line_num)
 			}
 			else if (equal(argv[i], "*")) {
 				rule->log_connect = 1;
+				rule->log_master = 1;
+				rule->log_cert = 1;
 				rule->log_content = 1;
 				rule->log_pcap = 1;
 #ifndef WITHOUT_MIRROR
@@ -2736,7 +2744,7 @@ filter_rule_parse(opts_t *opts, const char *name, char *value, int line_num)
 #ifndef WITHOUT_USERAUTH
 		"%s|"
 #endif /* !WITHOUT_USERAUTH */
-		"%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s"
+		"%s, action=%s|%s|%s|%s|%s, log=%s|%s|%s|%s|%s"
 #ifndef WITHOUT_MIRROR
 		"|%s"
 #endif /* !WITHOUT_MIRROR */
@@ -2751,7 +2759,7 @@ filter_rule_parse(opts_t *opts, const char *name, char *value, int line_num)
 #endif /* !WITHOUT_USERAUTH */
 		rule->all_sites ? "sites" : "",
 		rule->divert ? "divert" : "", rule->split ? "split" : "", rule->pass ? "pass" : "", rule->block ? "block" : "", rule->match ? "match" : "",
-		rule->log_connect ? "connect" : "", rule->log_content ? "content" : "", rule->log_pcap ? "pcap" : "",
+		rule->log_connect ? "connect" : "", rule->log_master ? "master" : "", rule->log_cert ? "cert" : "", rule->log_content ? "content" : "", rule->log_pcap ? "pcap" : "",
 #ifndef WITHOUT_MIRROR
 		rule->log_mirror ? "mirror" : "",
 #endif /* !WITHOUT_MIRROR */
@@ -2814,6 +2822,8 @@ opts_add_site(filter_site_t *site, filter_rule_t *rule)
 
 	// Multiple log actions can be set for the same site, hence bit-wise OR
 	s->log_connect |= rule->log_connect;
+	s->log_master |= rule->log_master;
+	s->log_cert |= rule->log_cert;
 	s->log_content |= rule->log_content;
 	s->log_pcap |= rule->log_pcap;
 #ifndef WITHOUT_MIRROR

src/opts.h

@@ -179,6 +179,8 @@ typedef struct filter_rule {
 
 	// Log action
 	unsigned int log_connect : 1;
+	unsigned int log_master : 1;
+	unsigned int log_cert : 1;
 	unsigned int log_content : 1;
 	unsigned int log_pcap : 1;
 #ifndef WITHOUT_MIRROR
@@ -209,6 +211,8 @@ typedef struct filter_site {
 	unsigned int block : 1;
 	unsigned int match : 1;
 	unsigned int log_connect : 1;
+	unsigned int log_master : 1;
+	unsigned int log_cert : 1;
 	unsigned int log_content : 1;
 	unsigned int log_pcap : 1;
 #ifndef WITHOUT_MIRROR

src/protossl.c

@@ -116,7 +116,7 @@ protossl_log_masterkey(pxy_conn_ctx_t *ctx, pxy_conn_desc_t *this)
 	// XXX: Remove ssl check? But the caller function is called by non-ssl protos.
 	if (this->ssl) {
 		/* log master key */
-		if (ctx->global->masterkeylog) {
+		if (ctx->log_master && ctx->global->masterkeylog) {
 			char *keystr;
 			keystr = ssl_ssl_masterkey_to_str(this->ssl);
 			if ((keystr == NULL) ||

src/proxy.c

@@ -175,6 +175,8 @@ proxy_conn_ctx_new(evutil_socket_t fd,
 	// Enable all logging for conn if proxyspec does not have any filter
 	if (!spec->opts->filter) {
 		ctx->log_connect = 1;
+		ctx->log_master = 1;
+		ctx->log_cert = 1;
 		ctx->log_content = 1;
 		ctx->log_pcap = 1;
 #ifndef WITHOUT_MIRROR

src/pxyconn.c

@@ -731,7 +731,7 @@ pxy_log_connect_src(pxy_conn_ctx_t *ctx)
 		pxy_log_connect_nonhttp(ctx);
 	}
 
-	if (ctx->src.ssl && ctx->global->certgendir) {
+	if (ctx->src.ssl && ctx->log_cert && ctx->global->certgendir) {
 		/* write SSL certificates to gendir */
 		protossl_srccert_write(ctx);
 	}
@@ -2030,6 +2030,14 @@ pxyconn_set_filter_action(pxy_conn_ctx_t *ctx, filter_site_t *site)
 		log_err_level_printf(LOG_INFO, "Site filter connect log for %s\n", site->site);
 		ctx->log_connect = 1;
 	}
+	if (site->log_master) {
+		log_err_level_printf(LOG_INFO, "Site filter master log for %s\n", site->site);
+		ctx->log_master = 1;
+	}
+	if (site->log_cert) {
+		log_err_level_printf(LOG_INFO, "Site filter cert log for %s\n", site->site);
+		ctx->log_cert = 1;
+	}
 	if (site->log_content) {
 		log_err_level_printf(LOG_INFO, "Site filter content log for %s\n", site->site);
 		ctx->log_content = 1;

src/pxyconn.h

@@ -325,6 +325,8 @@ struct pxy_conn_ctx {
 	// Default to all logging if no filter rules defined in proxyspec
 	// Otherwise, logging is disabled, so filter rules should enable each log action specifically
 	unsigned int log_connect : 1;
+	unsigned int log_master : 1;
+	unsigned int log_cert : 1;
 	unsigned int log_content : 1;
 	unsigned int log_pcap : 1;
 #ifndef WITHOUT_MIRROR

src/sslproxy.1

@@ -318,7 +318,7 @@ The syntax of filtering rules is as follows:
      uri (uri[*]|*)|
      ip (serveraddr|*)|
      *)]
-  [log ([connect] [content] [pcap] [mirror]|*)]
+  [log ([connect] [master] [cert] [content] [pcap] [mirror]|*)]
   |*)
 .LP
 The definition of which connections the rule action will be applied to is 
@@ -335,10 +335,14 @@ cn, and HTTP type of rules use host and uri site fields.
 connection.
 .LP
 If and how a connection should be logged is specified using the log part of 
-filtering rules. connect enables logging connection information to connect 
-log file, content enables logging packet contents to content log file, 
-pcap enables writing packets to pcap files, and mirror enables mirroring 
-packets to mirror interfaces or targets.
+filtering rules:
+.LP
+ - connect enables logging connection information to connect log file
+ - master enables logging of master keys
+ - cert enables logging of generated certificates
+ - content enables logging packet contents to content log file
+ - pcap enables writing packets to pcap files
+ - mirror enables mirroring packets to mirror interfaces or targets
 .LP
 For example, if the following rules are defined in a structured HTTPS proxyspec,
 .LP
@@ -376,8 +380,8 @@ Filtering rules are applied based on certain precedence orders:
  - The precedence of filter types is as Dst Host > SSL > HTTP.
  - The precedence of filter actions is as Divert > Split > Pass > Block. This is 
 only for the same type of filter rules.
- - The precedence of site fields is as sni > cn for ssl filter and host > uri 
-for http filter.
+ - The precedence of site fields is as sni > cn for SSL filter and host > uri 
+for HTTP filter.
 .LP
 For example, the pass action of a Dst Host filter rule is taken before the 
 split action of an SSL filter rule with the same from definition, due to the 
@@ -387,9 +391,10 @@ to the precedence order of site fields.
 .LP
 In terms of possible filter actions,
 .LP
- - Dst Host filter rules can take all of the actions.
- - SSL filter rules can take all of the actions.
- - HTTP filter rules can take the block action, but not divert, split, or pass 
+ - Dst Host filter rules can take all of the filter and log actions.
+ - SSL filter rules can take all of the filter and log actions.
+ - HTTP filter rules can take the match and block filter actions, but not 
+the divert, split, or pass actions. Also, HTTP filter rules cannot take log 
 actions.
 .LP
 Log actions do not configure any loggers. Global loggers for respective log 
@@ -403,7 +408,8 @@ You can append an asterisk * to site field of filtering rules for substring
 matching. Otherwise, the filter searches for an exact match with the site field 
 in the rule.
 .LP
-The order of from, to, and log parts is not important.
+The order of from, to, and log parts is not important. The order of log 
+actions is not important.
 .LP
 If the UserAuth option is disabled, only client IP addresses can be used in 
 the from part of filtering rules.

src/sslproxy.conf

@@ -286,11 +286,9 @@ PassUsers admin
 #      uri (uri[*]|*)|
 #      ip (serveraddr|*)|
 #      *)]
-#   [log ([connect] [content] [pcap] [mirror]|*)]
+#   [log ([connect] [master] [cert] [content] [pcap] [mirror]|*)]
 #   |*)
 #
-# Filter rules using host or uri can only take block action,
-# the other actions are ignored by issuing a warning.
 # PassSite example.com is equivalent to the following two Pass rules:
 # Pass to sni example.com
 # Pass to cn example.com

src/sslproxy.conf.5

@@ -344,7 +344,7 @@ The syntax of filtering rules is as follows:
      uri (uri[*]|*)|
      ip (serveraddr|*)|
      *)]
-  [log ([connect] [content] [pcap] [mirror]|*)]
+  [log ([connect] [master] [cert] [content] [pcap] [mirror]|*)]
   |*)
 .br
 
@@ -427,13 +427,7 @@ ValidateProto
 .br
 PassSite
 .br
-Divert filtering rules
-.br
-Split filtering rules
-.br
-Pass filtering rules
-.br
-Block filtering rules
+Divert|Split|Pass|Block|Match filtering rules
 .br
 \fB}\fR
 .br