Dynamic Black/Whitelists in your script.

This section contains scripts that hMailServer has contributed with. hMailServer 5 is needed to use these.
palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 00:48

Now with pagination. Change $no_of_records_per_page to whatever you want.

Code: Select all

<?php
$connect = mysqli_connect("localhost", "hmailserver", "supersecretpassword", "hmailserver");

if (isset($_GET['page'])) {
	$page = $_GET['page'];
	$display_pagination = 1;
} else {
	$page = 1;
	$total_pages = 1;
	$display_pagination = 0;
}

$no_of_records_per_page = 10;
$offset = ($page-1) * $no_of_records_per_page;
$total_pages_sql = "SELECT Count( * ) AS count FROM hm_mylists";
$result_rows = mysqli_query($connect,$total_pages_sql);
$total_rows = mysqli_fetch_array($result_rows)[0];
$total_pages = ceil($total_rows / $no_of_records_per_page);

$query = "SELECT * FROM hm_mylists ORDER BY id DESC LIMIT ".$offset.", ".$no_of_records_per_page;
$result = mysqli_query($connect,$query);

?>
<html>
 <head>
    <title>Live Table Data Edit Delete using Tabledit Plugin in PHP</title>
    <link rel="stylesheet" type="text/css" href="bootstrap.min.css" />
    <script src="jquery.min.js"></script>
    <script src="bootstrap.min.js"></script>
    <script src="jquery.tabledit.min.js"></script>
    </head>
    <body>
  <div class="container">
   <br />

<?php

if (isset($_GET['insert_function'])) {$insert_function = mysqli_real_escape_string($connect, $_GET['insert_function']);} else {$insert_function="";}
if (isset($_GET['insert_field'])) {$insert_field = mysqli_real_escape_string($connect, $_GET["insert_field"]);} else {$insert_field="";}
if (isset($_GET['insert_data'])) {$insert_data = mysqli_real_escape_string($connect, $_GET["insert_data"]);} else {$insert_data="";}
if (isset($_GET['insert_active'])) {$insert_active = mysqli_real_escape_string($connect, $_GET["insert_active"]);} else {$insert_active="";}

if ((!empty($insert_function)) && (!empty($insert_field)) && (!empty($insert_data))){
	$insert_query = "INSERT INTO hm_mylists (function,field,data,hits,lastused,active) VALUES ('".$insert_function."','".$insert_field."','".$insert_data."',0,NOW(),'".$insert_active."');";
	mysqli_query($connect, $insert_query);
	header("Location: ./index.php");
}

?>

    <table class="table">
     <thead>
      <tr>
       <th>Function</th>
       <th>Field</th>
       <th>Data</th>
       <th>Active</th>
       <th>Submit</th>
      </tr>
     </thead>
	 
	 
     <tbody>
		<tr>
			<form action='index.php' method='GET' onsubmit='return confirm(\"Are you sure you want to add this entry?\");'>
			<td>
				<input type='text' size='20' name='insert_function'>
			</td>
			<td>
				<input type='text' size='20' name='insert_field'>
			</td>
			<td>
				<input type='text' size='40' name='insert_data'>
			</td>
			<td>
				<select name='insert_active'>
					<option value=1>Yes</option>
					<option value=0>No</option>
				</select>
			</td>
			<td>
				<input type='submit' name='submit' value='Submit' >
			</td>
			</form>
		</tr>
     </tbody>
    </table>





   <br />
   <br />
            <div class="table-responsive">
    <h3 align="center">Live Table Data Edit Delete using Tabledit Plugin in PHP</h3><br />
    <table id="editable_table" class="table table-bordered table-striped">
     <thead>
      <tr>
       <th>ID</th>
       <th>Function</th>
       <th>Field</th>
       <th>Data</th>
       <th>Hits</th>
       <th>Last Used</th>
       <th>Active</th>
      </tr>
     </thead>
     <tbody>
     <?php
     while($row = mysqli_fetch_array($result))
     {
      echo '
      <tr>
       <td>'.$row["id"].'</td>
       <td>'.$row["function"].'</td>
       <td>'.$row["field"].'</td>
       <td>'.$row["data"].'</td>
       <td>'.$row["hits"].'</td>
       <td>'.$row["lastused"].'</td>
       <td>'.$row["active"].'</td>
      </tr>
      ';
     }
     ?>
     </tbody>
    </table>
<?php
if ($total_pages < 2){echo "";}
else {
	echo "<ul class='pagination'>";
	if($page <= 1){echo "<li class='page-item'><a class='page-link' href='#'>First </a></li>";} else {echo "<li class='page-item'><a class='page-link' href=\"index.php?page=1\">First </a><li>";}
	if($page <= 1){echo "<li><a class='page-link' href='#'>Prev </a></li class='page-item'>";} else {echo "<li class='page-item'><a class='page-link' href=\"index.php?page=".($page - 1)."\">Prev </a></li>";}
	if($page >= $total_pages){echo "<li class='page-item'><a class='page-link' href='#'>Next </a></li>";} else {echo "<li class='page-item'><a class='page-link' href=\"index.php?page=".($page + 1)."\">Next </a></li>";}
	if($page >= $total_pages){echo "<li class='page-item'><a class='page-link' href='#'>Last</a></li>";} else {echo "<li class='page-item'><a class='page-link' href=\"index.php?page=".$total_pages."\">Last</a></li>";}
	echo "</ul>";
}
if ($total_pages > 0){
	echo "<br />";
}
?>
   </div>
  </div>
 </body>
</html>
<script>
$(document).ready(function(){
     $('#editable_table').Tabledit({
      url:'action.php',
      columns:{
       identifier:[0, "id"],
       editable:[[1, 'function'], [2, 'field'], [3, 'data'], [6, 'active']]
      },
      restoreButton:false,
      onSuccess:function(data, textStatus, jqXHR)
      {
       if(data.action == 'delete')
       {
        $('#'+data.id).remove();
       }
      }
     });

});
 </script>

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-04-29 11:18

Did you check the last one i posted (framework.rar) ?

Direct edit in table, sorting, pagination and search... Checks all the boxes.
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 13:17

SorenR wrote:
2020-04-29 11:18
Did you check the last one i posted (framework.rar) ?

Direct edit in table, sorting, pagination and search... Checks all the boxes.
I got js errors right off the bat so I didn't try to figure out what was going on. I'll give it another try and see if I can get it working.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 13:30

Oh yeah, now that I'm looking at it again I remember the other reason: it needs a complete rewrite. :lol:

I'll see what I can do. Need some time. Today/tomorrow work is pretty busy. Also, remember, there's a gaping hole in my head for js, but looks like there's enough example there to figure it out. I just have to keep a reference copy unaltered. :mrgreen:

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-04-29 14:36

palinka wrote:
2020-04-29 13:30
Oh yeah, now that I'm looking at it again I remember the other reason: it needs a complete rewrite. :lol:

I'll see what I can do. Need some time. Today/tomorrow work is pretty busy. Also, remember, there's a gaping hole in my head for js, but looks like there's enough example there to figure it out. I just have to keep a reference copy unaltered. :mrgreen:
For reference ...
https://www.webslesson.info/2017/07/liv ... -ajax.html

I grabbed a copy of the js and css just to have them locally, they are also in the rar file :mrgreen:
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 18:02

Having trouble with this error. Not sure how to troubleshoot it. It was there before I touched it and still there.
Screenshot_20200429-120029_Brave.jpg

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-04-29 19:54

palinka wrote:
2020-04-29 18:02
Having trouble with this error. Not sure how to troubleshoot it. It was there before I touched it and still there.

Screenshot_20200429-120029_Brave.jpg
You have PM..
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 19:57

SorenR wrote:
2020-04-29 19:54
palinka wrote:
2020-04-29 18:02
Having trouble with this error. Not sure how to troubleshoot it. It was there before I touched it and still there.

Screenshot_20200429-120029_Brave.jpg
You have PM..
Got it. You didn't get any js errors? What about before you entered mysql credentials? Did you open the page before changing the credentials?

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 20:02

Was just playing with your demo. Yeah, it looks great. Very cool. I'll play with this thing tonight.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-04-29 20:06

palinka wrote:
2020-04-29 19:57
SorenR wrote:
2020-04-29 19:54
palinka wrote:
2020-04-29 18:02
Having trouble with this error. Not sure how to troubleshoot it. It was there before I touched it and still there.

Screenshot_20200429-120029_Brave.jpg
You have PM..
Got it. You didn't get any js errors? What about before you entered mysql credentials? Did you open the page before changing the credentials?
I first unpacked it on server and accessed it from browser (no database) and it came up no errors but blank cells...
The one you see I simply copied the directory from IIS on Windows 2003 Server R2 to Apache on my Synology NAS (Linux). Database remains on Windows 8)
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 20:11

SorenR wrote:
2020-04-29 20:06
palinka wrote:
2020-04-29 19:57
SorenR wrote:
2020-04-29 19:54


You have PM..
Got it. You didn't get any js errors? What about before you entered mysql credentials? Did you open the page before changing the credentials?
I first unpacked it on server and accessed it from browser (no database) and it came up no errors but blank cells...
The one you see I simply copied the directory from IIS on Windows 2003 Server R2 to Apache on my Synology NAS (Linux). Database remains on Windows 8)
That's weird. I got the same error on Brave mobile, Brave PC and MS Edge. I guess I'll just delete it and start over. I must have done something to break it at the very beginning.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-04-29 20:21

palinka wrote:
2020-04-29 20:11
SorenR wrote:
2020-04-29 20:06
palinka wrote:
2020-04-29 19:57


Got it. You didn't get any js errors? What about before you entered mysql credentials? Did you open the page before changing the credentials?
I first unpacked it on server and accessed it from browser (no database) and it came up no errors but blank cells...
The one you see I simply copied the directory from IIS on Windows 2003 Server R2 to Apache on my Synology NAS (Linux). Database remains on Windows 8)
That's weird. I got the same error on Brave mobile, Brave PC and MS Edge. I guess I'll just delete it and start over. I must have done something to break it at the very beginning.
All browsers except IE8 work flawlesly. Even my old Huawei P8 phone... On IE8 the page looks off but is useable.
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-29 23:09

Turned out to be a MySQL connection/credential (syntax) issue after all. Error gone now and the test data displays properly.

The rest should go easier. :roll:

Flukin' js... It was hiding the php errors.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-04-30 14:59

Something is going on that I cannot figure out. I hit a roadblock. More like the bridge over river kwai...

Download the attached. I keep getting the same json error. If you point your browser to fetch.php, it seems to render the json correctly. I "prettified" it for viewing. You can change that back in fetch.php by changing

echo json_encode($output, JSON_PRETTY_PRINT);

back to

echo json_encode($output);

Either way I get the same result. This has something to do with the js and I'm out of my league here. However, all the php parts work. Insert works. I assume update and delete work as well, but I can't test them because I can't get the data to display.

Another thing is, I had to change the MySQL column names because they conflict with js reserved words "function" and "data". So I changed the columns to xfunction, xfield and xdata. Keep that in mind when you test yours.

I've been over the js, but its not my thing and I'm just stumped.
Attachments
mylists.zip
(156.93 KiB) Downloaded 459 times

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-01 18:38

Working final version in case anyone wants.
Attachments
yourlists.zip
(159.95 KiB) Downloaded 459 times

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-01 19:18

palinka wrote:
2020-05-01 18:38
Working final version in case anyone wants.
Not working on my Linux/Apache since you converted to PDO :roll: Will spend some time later to investigate.

Anyways I just read that in Portland you can order a "Boober"... The norm is having "Uber" deliver food but Lucky Devil Lounge in Portland now have strippers deliver your food, optionally you can collect yourself as they also have a drive-thru strip tent for when you are waiting for your food.

"Food to GoGo"... And the best thing :!:

It started as a joke on Twitter :mrgreen:

The world is full of endless oportunities 8)
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-01 19:21

:mrgreen: :mrgreen: :mrgreen:

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-01 19:28

SorenR wrote:
2020-05-01 19:18
Not working on my Linux/Apache since you converted to PDO :roll: Will spend some time later to investigate.
Simple things: typo, spelling mistakes? Did you choose the correct "driver" in the config?

Php.ini:

extension=php_pdo_firebird.dll
extension=php_pdo_mysql.dll
extension=php_pdo_oci.dll
extension=php_pdo_odbc.dll
extension=php_pdo_pgsql.dll
extension=php_pdo_sqlite.dll

Pick your poison. :wink:

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-01 20:03

phpinfo() says...
Attachments
pdo.png
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-01 20:48

Try enabling extension=php_pdo_odbc.dll and using that.

Edit. Just thinking. Do you have SSL required on your mysql? There are no security settings in the connection strings.

Edit 2: security settings are database specific in mysql (I think) and you're using the hmailserver database, so it should be the default settings that hmailserver uses when you first install it.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-01 21:28

OK, I looked at your event handlers from the previous page (the first post changing from xml to db). You're using ODBC in eventhandlers.vbs, so enabling that in php.ini and using the ODBC driver should work. Not sure why the MySQL connection string isn't working. Is that maybe COM related? I know your db is on another machine. I don't know. I'm talking out my arse… :mrgreen: But we do know for sure that ODBC works for you.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-01 22:35

palinka wrote:
2020-05-01 21:28
OK, I looked at your event handlers from the previous page (the first post changing from xml to db). You're using ODBC in eventhandlers.vbs, so enabling that in php.ini and using the ODBC driver should work. Not sure why the MySQL connection string isn't working. Is that maybe COM related? I know your db is on another machine. I don't know. I'm talking out my arse… :mrgreen: But we do know for sure that ODBC works for you.
Well... I have two webservers... IIS on Windows has NO problems with PDO but access is blocked from Internet... Apache on LINUX have problems with PDO and is my public/general Webserver.

Roundcube, WordPress and some other stuff I have on Apache work perfectly with MySQL and MySQLi.

RoundCube on Apache/Linux forward COM requests to code on my IIS (Tunis made the drivers).

So... How should you know that :mrgreen:
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-01 22:46

SorenR wrote:
2020-05-01 22:35

So... How should you know that :mrgreen:
Well, I did say I was talking out my arse. Even the blind golfer hits a hole in one sometimes. :mrgreen:

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-02 03:09

Found the bugger...

Seems that the PHP version on my Linux is a bit old... The predefined constant "JSON_PRETTY_PRINT => 128" did not exist before PHP 5.40 and I'm using PHP 5.3.29. Unfortunately it is not possible to upgrade PHP on the (10 year old) NAS so I will probably end up buying a new.

I was getting "PHP Warning: json_encode() expects parameter 2 to be long" in "/var/log/httpd-error-user.log" ...

I replaced "JSON_PRETTY_PRINT" with 128 and that made it work. :mrgreen:
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-03 17:35

SorenR wrote:
2020-05-02 03:09
Found the bugger...

Seems that the PHP version on my Linux is a bit old... The predefined constant "JSON_PRETTY_PRINT => 128" did not exist before PHP 5.40 and I'm using PHP 5.3.29. Unfortunately it is not possible to upgrade PHP on the (10 year old) NAS so I will probably end up buying a new.

I was getting "PHP Warning: json_encode() expects parameter 2 to be long" in "/var/log/httpd-error-user.log" ...

I replaced "JSON_PRETTY_PRINT" with 128 and that made it work. :mrgreen:
Mine is getting a little long in the tooth too. Upgrading means upgrading horde, which I'm dreading. :evil:

ANYWAY.... I just got around to putting the db script into my eventhandlers. I tested with a bunch of messages and knocked out the stupid stuff, but there's one issue I can't figure out.

"ERROR" 13968 "2020-05-03 11:20:30.329" "Script Error: Source: Microsoft VBScript runtime error - Error: 800A1399 - Description: Syntax error in regular expression - Line: 126 Column: 6 - Code: (null)"

Code: Select all

Function oLookup(strRegEx, strMatch, bGlobal)
	If strRegEx = "" Then strRegEx = StrReverse(strMatch)
	With CreateObject("VBScript.RegExp")
		.Pattern = strRegEx
		.Global = bGlobal
		.MultiLine = True
		.IgnoreCase = True
		Set oLookup = .Execute(strMatch)   '<<---- LINE 126
	End With
End Function
I know its not the function, but I can't debug it because whatever syntax error is causing it is obviously getting redefined through the function. Any ideas where to look?

Is it possible that an empty field could cause this? For example, attempting to query against "List-Unsubscribe" when the List-Unsubscribe header does not exist?

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-03 21:21

palinka wrote:
2020-05-03 17:35
SorenR wrote:
2020-05-02 03:09
Found the bugger...

Seems that the PHP version on my Linux is a bit old... The predefined constant "JSON_PRETTY_PRINT => 128" did not exist before PHP 5.40 and I'm using PHP 5.3.29. Unfortunately it is not possible to upgrade PHP on the (10 year old) NAS so I will probably end up buying a new.

I was getting "PHP Warning: json_encode() expects parameter 2 to be long" in "/var/log/httpd-error-user.log" ...

I replaced "JSON_PRETTY_PRINT" with 128 and that made it work. :mrgreen:
Mine is getting a little long in the tooth too. Upgrading means upgrading horde, which I'm dreading. :evil:

ANYWAY.... I just got around to putting the db script into my eventhandlers. I tested with a bunch of messages and knocked out the stupid stuff, but there's one issue I can't figure out.

"ERROR" 13968 "2020-05-03 11:20:30.329" "Script Error: Source: Microsoft VBScript runtime error - Error: 800A1399 - Description: Syntax error in regular expression - Line: 126 Column: 6 - Code: (null)"

Code: Select all

Function oLookup(strRegEx, strMatch, bGlobal)
	If strRegEx = "" Then strRegEx = StrReverse(strMatch)
	With CreateObject("VBScript.RegExp")
		.Pattern = strRegEx
		.Global = bGlobal
		.MultiLine = True
		.IgnoreCase = True
		Set oLookup = .Execute(strMatch)   '<<---- LINE 126
	End With
End Function
I know its not the function, but I can't debug it because whatever syntax error is causing it is obviously getting redefined through the function. Any ideas where to look?

Is it possible that an empty field could cause this? For example, attempting to query against "List-Unsubscribe" when the List-Unsubscribe header does not exist?
I get no error if either "strRegEx" or "strMatch" is blank ("")...
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-03 21:47

Weird. For all I know it could be completely unrelated. I'll keep trying later. I'm still getting mail, so it's not critical for the moment.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-03 21:52

palinka wrote:
2020-05-03 21:47
Weird. For all I know it could be completely unrelated. I'll keep trying later. I'm still getting mail, so it's not critical for the moment.
Can you post the RegExp for your List-Unsubscribe?
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-04 14:54

Will do as soon as I sit down at my computer. But in the meantime I thought you'd be interested in this. I'm under attack since Friday night. Didn't even realize it. Fwban sweeping them away easy peasy.
Screenshot_20200504-084529_Brave.jpg
All password guessers. Helo for most of them is [127.0.0.1]
Screenshot_20200504-085100_Brave.jpg
You can have a look for yourself. The firewall ban demo mirrors what I'm actually blocking. It has its own database so there's no effect on my actual installation, so feel free to push buttons. :mrgreen:

https://firewallban.dynu.net/

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-04 15:25

SorenR wrote:
2020-05-03 21:52
palinka wrote:
2020-05-03 21:47
Weird. For all I know it could be completely unrelated. I'll keep trying later. I'm still getting mail, so it's not critical for the moment.
Can you post the RegExp for your List-Unsubscribe?
I only mentioned List-Unsubscribe because I know its a field that is not present in every message. Here's a selection of banhammers from OnAcceptMessage:

Code: Select all

	REM - Reject "List-Unsubscribe:"
	If oMessage.HeaderValue("List-Unsubscribe") <> "" Then
		strRegEx = MyListRegEx(MyListDict, "//Reject/List-Unsubscribe")
		Set oMatchCollection = oLookup(strRegEx, oMessage.HeaderValue("List-Unsubscribe"), False)
		For Each oMatch In oMatchCollection
			Result.Value = 2
			Result.Message = ". 08 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."
			Call Disconnect(oClient.IPAddress)
			Call FWBan(oClient.IPAddress, "ListUnsub-Rej", oClient.HELO, PTR_Record)
			Call AutoBan(oClient.IPAddress, "Rejected List Unsubscribe - " & oClient.HELO, 1, "h")
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "ListUnsub-Rej", oClient.IPAddress, oClient.HELO)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
			Call ReportToAbuseIPDB(oClient.IPAddress, "11", "Mail Rejected due to Proprietary Method on port " & oClient.Port & ", EHLO: " & oClient.HELO & "From: " & oMessage.FromAddress)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
			Exit Sub
		Next
	End If

	REM - Reject "X-Envelope-From:"
    strRegEx = MyListRegEx(MyListDict, "//Reject/X-Envelope-From")
    Set oMatchCollection = oLookup(strRegEx, oMessage.FromAddress, False)
    For Each oMatch In oMatchCollection
		Result.Value = 2
		Result.Message = ". 09 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."
		Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "X-Envelope-From", oClient.IPAddress, oClient.HELO)
		Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
		Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
		Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
		Call ReportToAbuseIPDB(oClient.IPAddress, "11", "Mail Rejected due to Proprietary Method on port " & oClient.Port & ", EHLO: " & oClient.HELO & "From: " & oMessage.FromAddress)
        Set oMatch = Nothing
        Set oMatchCollection = Nothing
		Exit Sub
	Next

	REM - Additional SPAM processing
	Dim Done : Done = False
	If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Then Done = True
	Do Until Done
		REM - Blacklist "IPRange:"
		strRegEx = MyListRegEx(MyListDict, "//Blacklist/IPRange")
		Set oMatchCollection = oLookup(strRegEx, oClient.IPAddress, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/IPRange = '" & oMatch.Value & "'", 5)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "BL-IP", oClient.IPAddress, oMatch.Value)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
			Exit Do
		Next
		REM - Blacklist "X-Envelope-From:"
		strRegEx = MyListRegEx(MyListDict, "//Blacklist/X-Envelope-From")
		Set oMatchCollection = oLookup(strRegEx, oMessage.FromAddress, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/X-Envelope-From = '" & oMatch.Value & "'", 5)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "BL-EnvFrom", oClient.IPAddress, oMatch.Value)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
			Exit Do
		Next
		REM - Blacklist "From:"
		strRegEx = MyListRegEx(MyListDict, "//Blacklist/From")
		Set oMatchCollection = oLookup(strRegEx, oMessage.From, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/From = '" & oMatch.Value & "'", 5)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "BL-From", oClient.IPAddress, oMatch.Value)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
			Exit Do
		Next
		REM - Blacklist "Subject:"
		strRegEx = MyListRegEx(MyListDict, "//Blacklist/Subject")
		Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "BL-Subject", oClient.IPAddress, oMatch.Value)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
			Exit Do
		Next
		REM - Blacklist Body (Plain Text)
		strRegEx = MyListRegEx(MyListDict, "//Blacklist/Bodytxt")
		Set oMatchCollection = oLookup(strRegEx, oMessage.Body, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/Body = '" & oMatch.Value & "'", 5)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "BL-Bodytxt", oClient.IPAddress, oMatch.Value)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
			Exit Do
		Next
		REM - Blacklist Body (HTML)
		REM - 
		REM -  <!-- ... -->   PHP: "(<!--[^>]*-->)"      JavaScript: "(<!--[\s\S]*?-->)"
		REM -  /*   ...  */   PHP: "(\/\*)[^>]*(\*\/)"   JavaScript: "(\/\*)[\s\S]*?(\*\/)"
		REM -  <!--[\\s\\S]*?(?:-->)?<!---+>?|<!(?![dD][oO][cC][tT][yY][pP][eE]|\\[CDATA\\])[^>]*>?|<[?][^>]*>?
		REM - 
		Dim strHTMLBody : strHTMLBody = oMessage.HTMLBody
		With CreateObject("VBScript.RegExp")
			.Pattern = "(\/\*[\s\S]*?\*\/)|(<[\s\S]*?>)"
			.Global = True
			.MultiLine = True
			.IgnoreCase = True
			strHTMLBody = .Replace(strHTMLBody, "")
		End With
		Set oMatchCollection = oLookup(strRegEx, strHTMLBody, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/Body = '" & oMatch.Value & "'", 5)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "Bodytxt", oClient.IPAddress, oClient.HELO)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgFrom", oClient.IPAddress, oMessage.FromAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgTo", oClient.IPAddress, oMessage.Recipients(0).OriginalAddress)
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "MsgSubject", oClient.IPAddress, oMessage.Subject)
			Exit Do
		Next
		Done = True
	Loop

	If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Then
		REM - Whitelist "X-Envelope-From:"
		strRegEx = MyListRegEx(MyListDict, "//Whitelist/X-Envelope-From")
		Set oMatchCollection = oLookup(strRegEx, oMessage.FromAddress, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call WhiteList(oMessage, "//Whitelist/X-Envelope-From = '" & oMatch.Value & "'")
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "Accepted", "WL-EnvFrom", oClient.IPAddress, oMatch.Value)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
		Next
		REM - Whitelist "From:"
		strRegEx = MyListRegEx(MyListDict, "//Whitelist/From")
		Set oMatchCollection = oLookup(strRegEx, oMessage.From, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call WhiteList(oMessage, "//Whitelist/From = '" & oMatch.Value & "'")
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "Accepted", "WL-From", oClient.IPAddress, oMatch.Value)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
		Next
		REM - Whitelist "Subject:"
		strRegEx = MyListRegEx(MyListDict, "//Whitelist/Subject")
		Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call WhiteList(oMessage, "//Whitelist/Subject = '" & oMatch.Value & "'")
			Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "Accepted", "WL-Subject", oClient.IPAddress, oMatch.Value)
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
		Next
	End If

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-04 15:39

Duhhhhh!!!! I looked at the Function MyListRegEx to see about writing to the event log. Of course it was already there. :oops:

Checked the event log and found this repeated many times.

12036 "2020-05-04 09:14:03.591" "ERROR: Empty string from MyListRegEx(Ransomeware, Bodytxt)"

I think I found it. You spelled "Ransomware" wrong, so I changed it throughout. Missed one. :oops: :oops: :oops:

Well, we found out one thing. If you're going to test against a "trunk" or "branch" in the database, you better make sure they exist and are spelled correctly. :mrgreen:

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-04 15:47

OOPs spoke too soon. I just sent a test message and got the same original error:

"ERROR" 12036 "2020-05-04 09:44:22.451" "Script Error: Source: Microsoft VBScript runtime error - Error: 800A1399 - Description: Syntax error in regular expression - Line: 126 Column: 2 - Code: (null)"

No error in the event log for this message. :(

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-04 16:39

Microsoft VBscript runtime error (0x800A1399) --> regular expression syntax error
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-04 22:28

SorenR wrote:
2020-05-04 16:39
Microsoft VBscript runtime error (0x800A1399) --> regular expression syntax error
See my code above. Everything looks OK to me. Maybe I screwed up something accidentally in the functions? I had to make changes to account for the db column name changes. Maybe I accidentally deleted something or whatever? Do you see anything?

I set up a test script with bare bones functionality. Function MyListRegEx seems to be working. I have some wscript.echos set up to see what works and the regex string appears to be working (several items from the db separated by pipe "|").

At this point, I think it may be simply vbs not being able to process certain regex types although I haven't narrowed it down to which entries are problematic.

https://stackoverflow.com/questions/347 ... e-800a1399

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-04 22:34

In my test script of Blacklist/Subject, I got the regex syntax error. As soon as I removed (deactivated) these 3 entries, the test script worked.

(^((?:yo|hi|sup|hello|greets|hey t?here)!?.?8?-?)?)$)
(^(cialis|viagara|erectile)(!?)(.?)(8?-?)?)?$)
(iPhone\s?(?:3G|4|5|6|SE|7|8|9|X|11)?(?:C|S|R)?\s?(?:Plus)?(?:Pro)?\s?(?:Max)?)

What is it about these entries that cause the regex to fail?

Can you replicate?

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-04 22:39

palinka wrote:
2020-05-04 22:34
In my test script of Blacklist/Subject, I got the regex syntax error. As soon as I removed (deactivated) these 3 entries, the test script worked.

(^((?:yo|hi|sup|hello|greets|hey t?here)!?.?8?-?)?)$)
(^(cialis|viagara|erectile)(!?)(.?)(8?-?)?)?$)
(iPhone\s?(?:3G|4|5|6|SE|7|8|9|X|11)?(?:C|S|R)?\s?(?:Plus)?(?:Pro)?\s?(?:Max)?)

What is it about these entries that cause the regex to fail?

Can you replicate?
... Hmm...
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-04 23:04

Code: Select all

(iPhone\s?(?:3G|4|5|6|SE|7|8|9|X|11)?(?:C|S|R)?\s?(?:Plus)?(?:Pro)?\s?(?:Max)?) - Confirmed.

(^((?:yo|hi|sup|hello|greets|hey t?here)!?.?8?-?)?)$) - Yours
^((?:yo|hi|sup|hello|greets|hey t?here)!?.?8?-?\)?)$ - Mine ... Confirmed

NB: "8?-?\)?" = 8-) You removed the "\" in "\)"

I presume it is the same case with this: 
(^(cialis|viagara|erectile)(!?)(.?)(8?-?)?)?$)

It really should be
(^(cialis|viagara|erectile)(!?)(.?)(8?-?\)?)?$)
Did you convert the database from XML ???
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-05 02:56

SorenR wrote:
2020-05-04 23:04
Did you convert the database from XML ???
Convert? What is this strange concept you speak of? :mrgreen:

Actually, I used the regex replace in notepad++ to change the xml entries to csv, then imported to MySQL using a powershell script. I assumed MySQL added any necessary escaping during the insert commands. I didn't really scrutinize it after it was added.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-05 03:08

SorenR wrote:
2020-05-04 23:04

Code: Select all

(iPhone\s?(?:3G|4|5|6|SE|7|8|9|X|11)?(?:C|S|R)?\s?(?:Plus)?(?:Pro)?\s?(?:Max)?) - Confirmed.
Did you convert the database from XML ???
OK, plugged this (above) in and manually escaped the "\" and now its working. :oops:

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-05 09:26

palinka wrote:
2020-05-05 03:08
SorenR wrote:
2020-05-04 23:04

Code: Select all

(iPhone\s?(?:3G|4|5|6|SE|7|8|9|X|11)?(?:C|S|R)?\s?(?:Plus)?(?:Pro)?\s?(?:Max)?) - Confirmed.
Did you convert the database from XML ???
OK, plugged this (above) in and manually escaped the "\" and now its working. :oops:
There are probably more of the same if I remember correctly. I did a manual conversion - changed the XML file in NP++ to become a CSV file and imported it into MySQL. Ended up with two "windows" comparing the XML version and the DB version.

I used this when logging the XML data to my database to make sure everything was escaped properly.

Code: Select all

Function xmlStats(strXML)
    Dim strSQL, oDB
    Set oDB = GetDatabaseObject
    strXML = Replace(strXML, "",     "\_")
    strXML = Replace(strXML, "\",    "\\")
    strXML = Replace(strXML, "%",    "\%")
    strXML = Replace(strXML, "'",    "\'")
    strXML = Replace(strXML, vbCr,   "\r")
    strXML = Replace(strXML, vbLf,   "\n")
    strXML = Replace(strXML, vbTab,  "\t")
    strXML = Replace(strXML, vbBack, "\b")
    strXML = Replace(strXML, Chr(34),"\" & Chr(34))
    strSQL = "INSERT INTO hm_xmlstat (created,timestamp,xmlnode,hits) VALUES (NOW(),NOW(),'" & strXML & "',1) ON DUPLICATE KEY UPDATE timestamp=NOW(),hits=(hits+1);"
    Call oDB.ExecuteSQL(strSQL)
    Set oDB = Nothing
End Function
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-05 14:07

SorenR wrote:
2020-05-05 09:26
There are probably more of the same if I remember correctly. I did a manual conversion - changed the XML file in NP++ to become a CSV file and imported it into MySQL. Ended up with two "windows" comparing the XML version and the DB version.

I used this when logging the XML data to my database to make sure everything was escaped properly.

Code: Select all

Function xmlStats(strXML)
    Dim strSQL, oDB
    Set oDB = GetDatabaseObject
    strXML = Replace(strXML, "",     "\_")
    strXML = Replace(strXML, "\",    "\\")
    strXML = Replace(strXML, "%",    "\%")
    strXML = Replace(strXML, "'",    "\'")
    strXML = Replace(strXML, vbCr,   "\r")
    strXML = Replace(strXML, vbLf,   "\n")
    strXML = Replace(strXML, vbTab,  "\t")
    strXML = Replace(strXML, vbBack, "\b")
    strXML = Replace(strXML, Chr(34),"\" & Chr(34))
    strSQL = "INSERT INTO hm_xmlstat (created,timestamp,xmlnode,hits) VALUES (NOW(),NOW(),'" & strXML & "',1) ON DUPLICATE KEY UPDATE timestamp=NOW(),hits=(hits+1);"
    Call oDB.ExecuteSQL(strSQL)
    Set oDB = Nothing
End Function
👍

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-05 21:45

I went back to my original php thing. I wasn't happy with how the js one jumps around. Edit something and it immediately goes back to the full list, so you have to search again, etc...

I made some changes. First of all, its really for mobile, so that's why the columns look the way they do. I also fixed up the drop down boxes to narrow down search options. Unfortunately, to get to the narrowed list of branch options, you first need to click search. Hey, its not js…

Also, edit/del is a new window. After you make edit or delete, the edit/del window closes and the main window, being the last one opened before edit/del, is what you go back to, but you need to refresh to see the changes you made. However, because it uses GET, refreshing the main window keeps you right where you were with respect to search, pagination, etc.

Anyway, I like it better. Mainly because its mobile friendly.
Attachments
black_white_admin_mobile.rar
(7.23 KiB) Downloaded 394 times

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-07 14:16

Do you see any difference between "Ransomware" and "Extortion"?

I guess the question is: is it worth the extra dozen or so lines of script to create a Function Extortion? I see them as being more or less the same.

Great idea, by the way, to pick up the SA headers for the extortion tags.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-05-07 15:30

palinka wrote:
2020-05-07 14:16
Do you see any difference between "Ransomware" and "Extortion"?

I guess the question is: is it worth the extra dozen or so lines of script to create a Function Extortion? I see them as being more or less the same.

Great idea, by the way, to pick up the SA headers for the extortion tags.
Ransomware was ment to catch ... well ... Ransom virus code.

Extortion I am nowhere near finished with yet. The filtering of charactersets and look-a-like lettering I am still working on.

Work in progress...

Code: Select all

Function TXTDecode(strTXT)
    '
    '   Decode HTMLBody
    '
    With CreateObject("VBScript.RegExp")
        .Global = True
        .MultiLine = True
        .IgnoreCase = True
        .Pattern =          "(&#x392;)|(\u0392)" : strTXT = .Replace(strTXT, "b")
        .Pattern = "(&#205;)|(&#x399;)|(\u0399)" : strTXT = .Replace(strTXT, "I")
        .Pattern =          "(&#x3B1;)|(\u03B1)" : strTXT = .Replace(strTXT, "a")
        .Pattern =          "(&#x3B2;)|(\u03B2)" : strTXT = .Replace(strTXT, "B")
        .Pattern = "(&#328;)|(&#x3B7;)|(\u03B7)" : strTXT = .Replace(strTXT, "n")
        .Pattern = "(&#237;)|(&#x3B9;)|(\u03B9)" : strTXT = .Replace(strTXT, "i")
        .Pattern =          "(&#x3BA;)|(\u03BA)" : strTXT = .Replace(strTXT, "k")
        .Pattern =          "(&#x3BD;)|(\u03BD)" : strTXT = .Replace(strTXT, "v")
        .Pattern =          "(&#x3BF;)|(\u03BF)" : strTXT = .Replace(strTXT, "o")
        .Pattern =          "(&#x3C1;)|(\u03C1)" : strTXT = .Replace(strTXT, "p")
        .Pattern =          "(&#x3C4;)|(\u03C4)" : strTXT = .Replace(strTXT, "t")
        .Pattern = "(&#252;)"                    : strTXT = .Replace(strTXT, "u")
        .Pattern = "(&#356;)"                    : strTXT = .Replace(strTXT, "T")
        .Pattern = "(\=[\r\n])"                  : strTXT = .Replace(strTXT, "")
        .Pattern = "(<font color=""white"">[a-z0-9]+<\/font>)" : strTXT = .Replace(strTXT, " ")
    End With
    TXTDecode = strTXT
End Function
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-05-07 15:35

SorenR wrote:
2020-05-07 15:30
Extortion I am nowhere near finished with yet. The filtering of charactersets and look-a-like lettering I am still working on.
Feels like Christmas getting near! :mrgreen:

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-08-16 04:41

FYI - bitcoin address regex here:

Code: Select all

^[13][a-km-zA-HJ-NP-Z0-9]{26,33}$

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-08-16 14:22

palinka wrote:
2020-08-16 04:41
FYI - bitcoin address regex here:

Code: Select all

^[13][a-km-zA-HJ-NP-Z0-9]{26,33}$
According to https://isc.sans.edu/forums/diary/Extra ... ils/23876/
Bitcoin addresses are base58check encoded integers with a checksum. The following regular expression will match a Bitcoin address:

\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b
https://en.bitcoin.it/wiki/Base58Check_encoding

What's with the {26,33} vs. {25,34} ?
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-08-17 04:45

SorenR wrote:
2020-08-16 14:22
What's with the {26,33} vs. {25,34} ?
I don't know, but I tested the one I found/posted on a "we're spying on you" spam and it picked up the bitcoin address.

I guess you couldn't go wrong with {25,34} for the purpose of checking for spam. Its not like we're actually validating the address.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2020-08-17 15:01

I have analysed my usage of "SnowShoe" and "LashBack" and found that Microsoft (mail-.+\.outbound\.protection\.outlook\.com) don't give a flying f**k about the "LashBack" list, they do however clean up records from the "SnowShoe" database. outlook.com servers were blocked/rejected/denied until one was found not in the "LashBack" database. At one point I thought I was under attack from outlook.com but it turned out that someone sent me 1 (ONE!) SPAM mail... :roll:

Giving it a good thought I have come to the conclusion that it would be the same problem for all "cloud based" domains like Yahoo and GMail.
First version was based on SPF data but required too many ressources. PTR info appeared simpler to use.

So ... In light of that I created a new Whitelist ...

INSERT INTO hm_mylists (trunk, branch, node) VALUES ('Whitelist', 'PTR', '(mail-.+\.outbound\.protection\.outlook\.com)');

If Not PTRWhitelist(oClient.IPAddress) Then "do stuff"

Code: Select all

Function PTRWhitelist(strIP) : PTRWhitelist = False
    Dim strLookup
    With CreateObject("DNSLibrary.DNSResolver")
        strLookup = .PTR(strIP)
    End With
    If strLookup = "" Then Exit Function
    Dim strRegEx, oMatch, oMatchCollection
    strRegEx = myListsRegEx(myListsDict, "//Whitelist/PTR")
    Set oMatchCollection = oLookup(strRegEx, strLookup, False)
    For Each oMatch In oMatchCollection
        Call myListsStat(myListsDict, oMatch)
        PTRWhitelist = True
    Next
    Set oMatch = Nothing
    Set oMatchCollection = Nothing
End Function
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-08-17 19:03

I did the same, but at the top of OnHELO. If whitelisted, then exit sub.

Also have a whitelist function:

Code: Select all

Function Whitelisted(strIP, strPort, strHELO) : Whitelisted = 0

	Dim a : a = Split(strIP, ".")
	Dim strLookup, strRegEx
	Dim IsWLMailSpike, IsWLHostKarma, IsWLNSZones, IsWLSPFBL, IsWLSpamDonkey, IsWLIPSWhitelisted
	
	With CreateObject("DNSLibrary.DNSResolver")
		strLookup = .DNSLookup(a(3) & "." & a(2) & "." & a(1) & "." & a(0) & ".rep.mailspike.net")
	End With
	strRegEx = "^127\.0\.0\.(18|19|20)$" '18=Good, 19=Very Good, 20=Excellent Reputation
	IsWLMailSpike = Lookup(strRegEx, strLookup)
	If IsWLMailSpike Then Call AccRejDB(0, strPort, "OnHELO", "Accepted", "WL-MailSpike", strIP, strHELO)

	With CreateObject("DNSLibrary.DNSResolver")
		strLookup = .DNSLookup(a(3) & "." & a(2) & "." & a(1) & "." & a(0) & ".hostkarma.junkemailfilter.com")
	End With
	strRegEx = "^127\.0\.0\.(1|5)$" '1=Good, 5=NoBL
	IsWLHostKarma = Lookup(strRegEx, strLookup)
	If IsWLHostKarma Then Call AccRejDB(0, strPort, "OnHELO", "Accepted", "WL-HostKarma", strIP, strHELO)

	With CreateObject("DNSLibrary.DNSResolver")
		strLookup = .DNSLookup(a(3) & "." & a(2) & "." & a(1) & "." & a(0) & ".wl.nszones.com")
	End With
	strRegEx = "^127\.0\.0\.5$" '5=whitelisted
	IsWLNSZones = Lookup(strRegEx, strLookup)
	If IsWLNSZones Then Call AccRejDB(0, strPort, "OnHELO", "Accepted", "WL-NSZones", strIP, strHELO)

	With CreateObject("DNSLibrary.DNSResolver")
		strLookup = .DNSLookup(a(3) & "." & a(2) & "." & a(1) & "." & a(0) & ".dnswl.spfbl.net")
	End With
	strRegEx = "^127\.0\.0\.(2|3|4|5)$" '2=excellent rep, 3=indispensable public service, 4=corp email (no marketing), 5=safe bulk mail
	IsWLSPFBL = Lookup(strRegEx, strLookup)
	If IsWLSPFBL Then Call AccRejDB(0, strPort, "OnHELO", "Accepted", "WL-SPFBL", strIP, strHELO)

	With CreateObject("DNSLibrary.DNSResolver")
		strLookup = .DNSLookup(a(3) & "." & a(2) & "." & a(1) & "." & a(0) & ".dnsbl.spamdonkey.com")
	End With
	strRegEx = "^126\.0\.0\.0$" '126.0.0.0=whitelisted
	IsWLSpamDonkey = Lookup(strRegEx, strLookup)
	If IsWLSpamDonkey Then Call AccRejDB(0, strPort, "OnHELO", "Accepted", "WL-SpamDonkey", strIP, strHELO)

	With CreateObject("DNSLibrary.DNSResolver")
		strLookup = .DNSLookup(a(3) & "." & a(2) & "." & a(1) & "." & a(0) & ".ips.whitelisted.org")
	End With
	strRegEx = "^127\.0\.0\.2$" '2=whitelisted
	IsWLIPSWhitelisted = Lookup(strRegEx, strLookup)
	If IsWLIPSWhitelisted Then Call AccRejDB(0, strPort, "OnHELO", "Accepted", "WL-IPSWhitelisted", strIP, strHELO)

	If (IsWLMailSpike OR IsWLHostKarma OR IsWLNSZones OR IsWLSPFBL OR IsWLSpamDonkey OR IsWLIPSWhitelisted) Then Whitelisted = 1

End Function
Then I use if IsWhitelisted to skip certain spam tests, like geoip. I don't mind banning the world, but once in a blue moon a legit message comes in from a banned country and I still want to receive it. :mrgreen:

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2020-10-09 16:05

palinka wrote:
2020-08-16 04:41
FYI - bitcoin address regex here:

Code: Select all

^[13][a-km-zA-HJ-NP-Z0-9]{26,33}$
Update.

Code: Select all

\b((bc|tb)(0([ac-hj-np-z02-9]{39}|[ac-hj-np-z02-9]{59})|1[ac-hj-np-z02-9]{8,87})|([13]|[mn2])[a-km-zA-HJ-NP-Z1-9]{25,39})\b\s
From: https://stackoverflow.com/questions/216 ... -addresses

To test: https://regex101.com/r/Gu5p9p/1

The test stuff came right from the stackoverflow page. All I did was try it out there and create a link to it.

The \s at the end seems to be important in avoiding false positives. Word boundary alone does not seem to do it. The example presented as FP was randomized file names in a URL. In a BTC scam, there will always be a space at the end because the spammer does not want you to screw up his money address by adding a period, comma, dash, or any other punctuation or anything else. The same would hold true for BTC addresses in legit mail.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2021-10-15 13:13

Please help me Soren-Wan! You're my only hope!

I broke something and I can't figure it out because I'm poor with scripting dictionary.. :( (I think)

Symptoms:
1) Send email with blacklisted subject from gmail to my hmailserver account
2) Message does not get blacklisted (X-hMailServer-Reason-Score: 0)
3) "Hits" does not get updated in the database
4) Nothing reported in event log (no db conn errors)
5) Error log contains:

Code: Select all

"ERROR"	3476	"2021-10-15 06:49:33.425"	"Script Error: Source: Microsoft VBScript runtime error - Error: 800A0009 - Description: Subscript out of range: 'i' - Line: 583 Column: 16 - Code: (null)"
My code (your code):

Code: Select all

Function MyListRegEx(MyListDict, MyNode) : MyListRegEx = ""
    Dim i, a, strData, sqlDBDrv
    Dim oRecord, oDB : Set oDB = CreateObject("ADODB.Connection")
    oDB.Open "Driver={MariaDB ODBC 3.1 Driver}; Server=localhost; Database=hmailserver; User=hmailserver; Password=supersecretpassword;"
    If oDB.State <> 1 Then
        EventLog.Write( "MyListRegEx - ERROR: Could not connect to database" )
        MyListRegEx = "VOID"
        Exit Function
    End If
    MyListDict.RemoveAll
    a = Split(MyNode, "/")
    Set oRecord = oDB.Execute("SELECT * FROM " & DBTABLE & " WHERE active = 1 AND trunk = '" & a(2) & "' AND branch = '" & a(3) & "';")
    Do Until oRecord.EOF
        If (Trim(oRecord("node")) <> "") Then
            strData = strData & Trim(oRecord("node")) & "|"
            MyListDict.Add CStr(oRecord("id")), CStr(oRecord("node"))
        End If
        oRecord.MoveNext
    Loop
    If (Trim(strData) <> "") Then
        MyListRegEx = Left(strData,Len(strData)-1)
    Else
        EventLog.Write( "ERROR: Empty string from MyListRegEx(" & a(2) & ", " & a(3) & ")" )
        MyListRegEx = "VOID"
    End If
    Set oRecord = Nothing
    oDB.Close
    Set oDB = Nothing
End Function

Function MyListStat(MyListDict, oMatch)
    Dim i, a
    Dim strSQL, oDB : Set oDB = GetDatabaseObject
    a = MyListDict.Keys
    If oMatch.SubMatches.Count > 0 Then
        For i = 0 To oMatch.SubMatches.Count - 1
            If oMatch.SubMatches(i) <> Empty Then
                strSQL = "UPDATE " & DBTABLE & " SET tracked = NOW(), hits = (hits + 1) WHERE id = " & a(i) & ";"        '<<<--- LINE 583
                Call oDB.ExecuteSQL(strSQL)
                Exit For
            End If
        Next
    End If
    Set oDB = Nothing
End Function

Sub BlackList(oMessage, strMatch, iScore)
    Dim i, Done : Done = False
    If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Then
        i = CInt(oMessage.HeaderValue("X-hMailServer-Reason-Score"))
    Else
        oMessage.HeaderValue("X-hMailServer-Spam") = "YES"
        i = 0
    End If
    oMessage.HeaderValue("X-hMailServer-Reason-0") = "BlackListed - (Score: " & iScore & " )"
    oMessage.HeaderValue("X-hMailServer-Reason-Score") = iScore + i
    i = 1
    Do Until Done
        If (oMessage.HeaderValue("X-hMailServer-BlackList-" & i) = "") Then
            oMessage.HeaderValue("X-hMailServer-BlackList-" & i) = strMatch
            Exit Do
        Else
            i = i + 1
        End If
    Loop
    oMessage.Save
End Sub

Sub OnAcceptMessage(oClient, oMessage)

	' left out Dim's, other operations, etc for brevity

	REM - Blacklist "Subject:"
	strRegEx = MyListRegEx(MyListDict, "//Blacklist/Subject")
	Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
	For Each oMatch In oMatchCollection
		Call MyListStat(MyListDict, oMatch)
		Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
		Set oMatch = Nothing
		Set oMatchCollection = Nothing
	Next

End Sub
I can't figure it out. I noticed that it wasn't working when some list mail that was whitelisted > X-Envelope-From was not working. I set up the simple test for blacklisting because I can easily send a test message with offending subject line.

It looks like its finding the matching string, but I can't figure out why its BOTH not updating the table (MyListStat) AND blacklisting and whitelisting are not working. I'm not even sure if they're connected or not. However, I do know that the blacklist function works when called from operations other than the dynamic black/white list. For example, I have one filter that looks for received header gmailapi.google.com which successfully blacklists those messages. Anyway, I'm totally lost here and would greatly appreciate a looksie.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2021-10-15 14:01

Code: Select all

Sub OnAcceptMessage(oClient, oMessage)

	' left out Dim's, other operations, etc for brevity

	REM - Blacklist "Subject:"
	strRegEx = MyListRegEx(MyListDict, "//Blacklist/Subject")
	If strRegEx <> "VOID" Then
		Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
		Next
	End If
	Set oMatch = Nothing
	Set oMatchCollection = Nothing

End Sub
Don't "Nothing" your objects (oMatch/oMatchCollection) unless you are done using them ;-)

Also, MyListRegEx() returns "VOID" for a reason ...
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2021-10-15 16:45

SorenR wrote:
2021-10-15 14:01

Code: Select all

Sub OnAcceptMessage(oClient, oMessage)

	' left out Dim's, other operations, etc for brevity

	REM - Blacklist "Subject:"
	strRegEx = MyListRegEx(MyListDict, "//Blacklist/Subject")
	If strRegEx <> "VOID" Then
		Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
		For Each oMatch In oMatchCollection
			Call MyListStat(MyListDict, oMatch)
			Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
		Next
	End If
	Set oMatch = Nothing
	Set oMatchCollection = Nothing

End Sub
Don't "Nothing" your objects (oMatch/oMatchCollection) unless you are done using them ;-)

Also, MyListRegEx() returns "VOID" for a reason ...
Yikes. I tried that and its still not working. Same error "Subscript out of range: 'i' - Line: 583"

Any ideas?

Code: Select all

	REM - Blacklist "Subject:"
	strRegEx = MyListRegEx(MyListDict, "//Blacklist/Subject")
	Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
	For Each oMatch In oMatchCollection
		Call MyListStat(MyListDict, oMatch)
		Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
		Exit Sub
	Next

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2021-10-15 19:05

Hmm...

This is cut'n pasted directly from my eventhandlers ... with your DB configs ...

This works ;-)

Code: Select all

'******************************************************************************************************************************
'********** Configuration                                                                                            **********
'******************************************************************************************************************************
'
'   MySQL
'
Private Const DBDRVR     = "MariaDB ODBC 3.1 Driver"
Private Const DBSERVER   = "localhost"
Private Const DBPORT     = "3306"
Private Const DBNAME     = "hmailserver"
Private Const DBEXTRA    = "hmailserver_extra"
Private Const DBUID      = "hmailserver"
Private Const DBPW       = #COSMIC_SECRET
'
'   Dynamic Black/White lists
'
Private Const DBTABLE    = "hm_mylists"
'
'   Global dictionaries
'
Private myListsDict : Set myListsDict = CreateObject("Scripting.Dictionary")

'******************************************************************************************************************************
'********** Dynamic Black/White lists                                                                                **********
'******************************************************************************************************************************

'   hm_mylists  CREATE TABLE hm_mylists (
'                 id        bigint(20)   NOT NULL AUTO_INCREMENT,
'                 trunk     varchar(255) NOT NULL,
'                 branch    varchar(255) NOT NULL,
'                 node      varchar(255) NOT NULL,
'                 hits      bigint(20)   NOT NULL DEFAULT '0',
'                 tracked   datetime     NOT NULL,
'                 active    tinyint(1)   NOT NULL DEFAULT '1',
'                 PRIMARY KEY (id)
'               ) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Function myListsRegEx(myListsDict, MyNode)
    Dim i, a, strData
    Dim oRecord, oDB : Set oDB = CreateObject("ADODB.Connection")
    oDB.Open "DRIVER={" & DBDRVR & "};Server=" & DBSERVER & ";Port=" & DBPORT & ";Database=" & DBNAME & ";Uid=" & DBUID & ";Pwd=" & DBPW & ";FOUND_ROWS=1;"
    If oDB.State <> 1 Then
        EventLog.Write( "myListsRegEx - ERROR: Could not connect to database" )
        myListsRegEx = "VOID"
        Exit Function
    End If
    myListsDict.RemoveAll
    a = Split(MyNode, "/")
    Set oRecord = oDB.Execute("SELECT * FROM " & DBTABLE & " WHERE active = 1 AND trunk = '" & a(2) & "' AND branch = '" & a(3) & "';")
    Do Until oRecord.EOF
        If (Trim(oRecord("node")) <> "") Then
            strData = strData & Trim(oRecord("node")) & "|"
            myListsDict.Add CStr(oRecord("id")), CStr(oRecord("node"))
        End If
        oRecord.MoveNext
    Loop
    If (Trim(strData) <> "") Then
        myListsRegEx = Left(strData,Len(strData)-1)
    Else
        myListsRegEx = "VOID"
    End If
    Set oRecord = Nothing
    oDB.Close
    Set oDB = Nothing
End Function

Function myListsStat(myListsDict, oMatch)
    Dim i, a
    Dim strSQL, oDB : Set oDB = CreateObject("ADODB.Connection")
    oDB.Open "DRIVER={" & DBDRVR & "};Server=" & DBSERVER & ";Port=" & DBPORT & ";Database=" & DBNAME & ";Uid=" & DBUID & ";Pwd=" & DBPW & ";FOUND_ROWS=1;"
    If oDB.State <> 1 Then
        EventLog.Write( "myListsRegEx - ERROR: Could not connect to database" )
        myListsRegEx = "VOID"
        Exit Function
    End If
    a = myListsDict.Keys
    If oMatch.SubMatches.Count > 0 Then
        For i = 0 To oMatch.SubMatches.Count-1
            If Not IsEmpty(oMatch.SubMatches(i)) Then
                strSQL = "UPDATE " & DBTABLE & " SET tracked = NOW(), hits = (hits + 1) WHERE id = " & a(i) & ";"
                Call oDB.Execute(strSQL)
                Exit For
            End If
        Next
    End If
    Set oDB = Nothing
End Function

'******************************************************************************************************************************
'********** Subroutines                                                                                              **********
'******************************************************************************************************************************

Sub BlackList(oMessage, strMatch, iScore)
    Dim i, Done : Done = False
    If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Then
        i = CInt(oMessage.HeaderValue("X-hMailServer-Reason-Score"))
    Else
        oMessage.HeaderValue("X-hMailServer-Spam") = "YES"
        i = 0
    End If
    oMessage.HeaderValue("X-hMailServer-Blacklist") = "YES"
    oMessage.HeaderValue("X-hMailServer-Reason-0") = "BlackListed - (Score: " & iScore & ")"
    oMessage.HeaderValue("X-hMailServer-Reason-Score") = iScore + i
    i = 1
    Do Until Done
        If (oMessage.HeaderValue("X-hMailServer-BlackList-" & i) = "") Then
            oMessage.HeaderValue("X-hMailServer-BlackList-" & i) = strMatch
            Exit Do
        Else
            i = i + 1
        End If
    Loop
    oMessage.Save
End Sub

'******************************************************************************************************************************
'********** hMailServer Triggers                                                                                     **********
'******************************************************************************************************************************

Sub OnAcceptMessage(oClient, oMessage)

    '   STUFF !!

    '
    '   Blacklists
    '
    Done = False
    If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Or Whitelisted Then Done = True
    Do Until Done

        '   MORE STUFF !!

        '
        '   Blacklist "Subject:"
        '
        strRegEx = myListsRegEx(myListsDict, "//Blacklist/Subject")
        If strRegEx <> "VOID" Then
            Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
            For Each oMatch In oMatchCollection
                Call myListsStat(myListsDict, oMatch)
                Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
            Next
            If oMatchCollection.Count > 0 Then Exit Do
        End If

        '   MORE STUFF !!

        Exit Do
    Loop

    '   MORE STUFF !!

End Sub
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2021-10-15 20:06

Hmmm... copied it verbatim except for the connection strings. Still no dice. Same error: Subscript out of range: 'i' in the MyListStat function:

Code: Select all

strSQL = "UPDATE " & DBTABLE & " SET tracked = NOW(), hits = (hits + 1) WHERE id = " & a(i) & ";"
If MyListStat is being called, then obviously the script is picking up the regex searches. So with that error, a) the db doesn't get updated in MyListStat() and it dies and exits the event, I guess, because Blacklist() doesn't get called either.

I just rearranged the order of what to call so that Blacklist() gets called first:

Code: Select all

		REM - Blacklist "Subject:"
		strRegEx = MyListRegEx(MyListDict, "//Blacklist/Subject")
		If strRegEx <> "VOID" Then
			Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
			For Each oMatch In oMatchCollection
				Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
				Call MyListStat(MyListDict, oMatch)
				Exit Do
			Next
		End If
This results in the message getting blacklisted as it should. YAY! At least I eliminated blacklist() as a potential cause.

Still, I got the same error (Subscript out of range: 'i'). "i" can't be empty due to the If statement around the db update.

The full error is:

Code: Select all

"Script Error: Source: Microsoft VBScript runtime error - Error: 800A0009 - Description: Subscript out of range: 'i' - Line: 590 Column: 4 - Code: (null)"
Empty & null are not the same in vbs? Anyway, it shouldn't be null or empty because there was a bona fide regex match sent along in var oMatch. I'm stumped.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dynamic Black/Whitelists in your script.

Post by SorenR » 2021-10-15 20:57

What's in line 590 +/- 5 lines ?

Can you verify there are listings in the database that matches and active=1 ??
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2021-10-16 02:56

SorenR wrote:
2021-10-15 20:57
What's in line 590 +/- 5 lines ?

Can you verify there are listings in the database that matches and active=1 ??
Lines 577-597:

Code: Select all

Function MyListStat(MyListDict, oMatch)
	Dim i, a
	Dim strSQL, oDB : Set oDB = CreateObject("ADODB.Connection")
	oDB.Open "Driver={MariaDB ODBC 3.1 Driver}; Server=localhost; Database=hmailserver; User=hmailserver; Password=supersecretpassword;"
	If oDB.State <> 1 Then
		EventLog.Write( "myListsRegEx - ERROR: Could not connect to database" )
		myListsRegEx = "VOID"
		Exit Function
	End If
	a = myListDict.Keys
	If oMatch.SubMatches.Count > 0 Then
		For i = 0 To oMatch.SubMatches.Count-1
			If Not IsEmpty(oMatch.SubMatches(i)) Then
				strSQL = "UPDATE " & DBTABLE & " SET tracked = NOW(), hits = (hits + 1) WHERE id = " & a(i) & ";"   '<<-- LINE 590
				Call oDB.Execute(strSQL)
				Exit For
			End If
		Next
	End If
	Set oDB = Nothing
End Function
The message i was using to test contained subject line: Re[5]: gettin fluke off niagara beans

(I didn't want any false positives :lol: )

And the database entry for this does exist:
db.png

Edit - I have "MyListStat" (vs "MyListsStat") because that's how you had it originally back in the day. Too many lines calling the function to change easily. :D

Also, the timestamp in the image is today because I edited it using my php admin, which updates "tracked" when edited.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2021-10-16 03:25

I added some debugging.

Code: Select all

Function MyListStat(MyListDict, oMatch)
	Dim i, a
	Dim strSQL, oDB : Set oDB = CreateObject("ADODB.Connection")
	oDB.Open "Driver={MariaDB ODBC 3.1 Driver}; Server=localhost; Database=hmailserver; User=hmailserver; Password=supersecretpassword;"
	If oDB.State <> 1 Then
		EventLog.Write( "myListsRegEx - ERROR: Could not connect to database" )
		myListsRegEx = "VOID"
		Exit Function
	End If
	a = myListDict.Keys
	If oMatch.SubMatches.Count > 0 Then
		For i = 0 To oMatch.SubMatches.Count-1
			If Not IsEmpty(oMatch.SubMatches(i)) Then
				EventLog.Write("i       : " & i)
				EventLog.Write("submatch: " & a(i))
				strSQL = "UPDATE " & DBTABLE & " SET tracked = NOW(), hits = (hits + 1) WHERE id = " & a(i) & ";"
				Call oDB.Execute(strSQL)
				Exit For
			End If
		Next
	End If
	Set oDB = Nothing
End Function
Result: still erroring, but at least it captured the first one. From eventlog:

Code: Select all

3476	"2021-10-15 21:12:21.741"	"i       : 11"
Next, I queried the db to see what would happen. Test message is Blacklist/Subject.

Code: Select all

SELECT * FROM hm_black_white_list WHERE active = 1 AND trunk = 'BlackList' AND branch = 'Subject'
Result: 7 rows.

Now we're getting somewhere. MyListDict must be null at interval 11 becuase there are only 7 blacklist/subject entries in total. I assume myListDict should be cleared somewhere, but where/when? "MyListDict.RemoveAll" is part of MyListRegEx(). Maybe its not working or maybe its in the wrong location?

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Dynamic Black/Whitelists in your script.

Post by palinka » 2021-10-16 04:31

Last one for tonight. I changed the debugging to be at the top of the for loop. As above there are 7 rows as mysql query result. Plus, I changed "For i = 0 To oMatch.SubMatches.Count-1" to "For i = 0 To myListDict.Count-1" which got rid of the out of range error.

Code: Select all

Function MyListStat(MyListDict, oMatch)
	Dim i, a
	Dim strSQL, oDB : Set oDB = CreateObject("ADODB.Connection")
	oDB.Open "Driver={MariaDB ODBC 3.1 Driver}; Server=localhost; Database=hmailserver; User=hmailserver; Password=SSnGLBs8XswL2r0h;"
	If oDB.State <> 1 Then
		EventLog.Write( "myListsRegEx - ERROR: Could not connect to database" )
		myListsRegEx = "VOID"
		Exit Function
	End If
	a = myListDict.Keys
	EventLog.Write("key count  : " & myListDict.Count)
	EventLog.Write("match count: " & oMatch.SubMatches.Count)
	If oMatch.SubMatches.Count > 0 Then
		For i = 0 To myListDict.Count-1
			EventLog.Write("i        : " & i)
			EventLog.Write("key      : " & a(i))
			EventLog.Write("submatch : " & oMatch.SubMatches(i))
			If Not IsEmpty(oMatch.SubMatches(i)) Then
				strSQL = "UPDATE " & DBTABLE & " SET tracked = NOW(), hits = (hits + 1) WHERE id = " & a(i) & ";"
				Call oDB.Execute(strSQL)
				Exit For
			End If
		Next
	End If
	Set oDB = Nothing
End Function
Result:

Code: Select all

3476	"2021-10-15 22:25:02.265"	"key count  : 7"
3476	"2021-10-15 22:25:02.266"	"match count: 12"
3476	"2021-10-15 22:25:02.267"	"i        : 0"
3476	"2021-10-15 22:25:02.267"	"key      : 51"
3476	"2021-10-15 22:25:02.268"	"submatch : "
3476	"2021-10-15 22:25:02.268"	"i        : 1"
3476	"2021-10-15 22:25:02.269"	"key      : 52"
3476	"2021-10-15 22:25:02.269"	"submatch : "
3476	"2021-10-15 22:25:02.270"	"i        : 2"
3476	"2021-10-15 22:25:02.270"	"key      : 158"
3476	"2021-10-15 22:25:02.271"	"submatch : "
3476	"2021-10-15 22:25:02.271"	"i        : 3"
3476	"2021-10-15 22:25:02.271"	"key      : 189"
3476	"2021-10-15 22:25:02.272"	"submatch : "
3476	"2021-10-15 22:25:02.272"	"i        : 4"
3476	"2021-10-15 22:25:02.273"	"key      : 203"
3476	"2021-10-15 22:25:02.273"	"submatch : "
3476	"2021-10-15 22:25:02.274"	"i        : 5"
3476	"2021-10-15 22:25:02.274"	"key      : 206"
3476	"2021-10-15 22:25:02.275"	"submatch : "
3476	"2021-10-15 22:25:02.275"	"i        : 6"
3476	"2021-10-15 22:25:02.275"	"key      : 219"
3476	"2021-10-15 22:25:02.276"	"submatch : "

So its not the myListDict key count that is wrong. Its the match count.

I tried setting oMatch to nothing like this:

Code: Select all

Sub OnAcceptMessage(oClient, oMessage)
	'
	' STUFF
	'
	REM - Additional SPAM processing
	Dim Done : Done = False
	If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Then Done = True
	Do Until Done
		REM - Blacklist "Subject:"
		strRegEx = MyListRegEx(MyListDict, "//Blacklist/Subject")
		If strRegEx <> "VOID" Then
			Set oMatchCollection = oLookup(strRegEx, oMessage.Subject, False)
			For Each oMatch In oMatchCollection
				Call BlackList(oMessage, "//Blacklist/Subject = '" & oMatch.Value & "'", 5)
				Call MyListStat(MyListDict, oMatch)
				Call AccRejDB(msgID, oClient.Port, "OnAcceptMessage", "REJECTED", "BL-Subject", oClient.IPAddress, "Matching string: " & oMatch.Value)
				Exit Do
			Next
			Set oMatch = Nothing
			Set oMatchCollection = Nothing
		End If
		'
		' MOAR STUFF
		'
		Exit Do
	Loop
	'
	' MOAR STUFF
	'
End Sub
This did not work. There are no vbs errors, but with the submatch always null, it does not update the database.

Post Reply