Valid email addresses being rejected as spam

[aschaevitz]

I have a form that has several fields for various email addresses (type="text") and is processed by the tectite formmail script (v.8.14). A user is being redirected to a page that states:

"Your form submission has been rejected as it appears to be an abuse of our server (ca.mousesquad.org).
Our supplier of forms processing software has provided more information about this error.

The following input looks like a junk attack on our server. Please avoid scientific or technical terms with long sequences of consonants or vowels: <email address>,<email address>"

That's the Santa Barbara McKinley School email address. The same thing is happening with address that end with @sbsdk12.org (that's the Santa Barbara School District K-12 email server). Clearly, these are valid email addresses.

Is this a bug in the script or am I not setting some parameter correctly?

[aschaevitz]

Interestingly ( and adding complexity), I looked at recent forms that were successfully sent/received. They included addresses such as @kippsummit.org, @lareina.com, @sandi.net, @pacificasd.org, @fmsd.org, @salinas.k12.ca.us and @hemetusd.k12.ca.us. So, it is not at all clear why email addresses ending in @sbmckinley.org and sbsdk12.org should have been rejected.

[russellr]

Hi,

I don't think it's a bug, because we avoid checking against any fields that might have this type of information.

What's the URL for your HTML form, so I can check it out?

[aschaevitz]

The URL is http://ca.mousesquad.org/forms/squadapplication.php

It seems that most email addresses pose no problem but some do, without any clear textual difference between them. I note that the first email address that was in the pasted error text didn't display. It was @sbmckinley.org and all the email addresses prior to the "@" were in the form <first initial><last name> so were not unusual character strings.

[russellr]

Hi,

The reason an address at "@sbmckinley.org" is caught, is that "sbmck" is 5 consecutive consonants.

Does the problem occur only when you type these email addresses into a field while filling in your form?

If not, then I'll need to investigate further.

If so, and it's the "Email" field, then that's the cause of the problem.

"Email" isn't a FormMail Special Field, so it's subject to junk checking.

However, your deriving of "email" from "Email" is one of our suggestions to overcome a little annoyance with fmbadhandler.

Therefore, you can't just rename "Email" to "email" - which would stop the problem from happening - because it would cause a problem when returning to your form from fmbadhandler.

Changing the junk detection to check for 6 or more consonants will avoid it, but that's not the right solution.

The right solution is for you to be able to configure FormMail so that certain fields are skipped in the junk detection.

Email addresses, URLs, and other things are very likely to look like junk.

So, the fields on your form that can contain these things and are not FormMail Special Fields, need to be identified and skipped for this type of attack detection.

Please confirm the question above, and if I'm right about the problem, I'll look into solving it ASAP.

[aschaevitz]

The form field for all email addresses in all my forms are of the structure:
<input type="text" name="(name)" id="(id)" size="25" />

So far, the only indication I have had from users relates to email address fields causing rejection by FormMail. That's not to say it hasn't happened in other fields, but it would be highly unlikely given the form contents. Even with email fields, this problem is relatively rare (most email addresses, I guess, don't have 5 or more consecutive consonants). However, clearly I have users whose email address fit that description and I must accommodate that.

Your sentence "However, your deriving of "email" from "Email" is one of our suggestions to overcome a little annoyance with fmbadhandler." is confusing and I'm not sure what you mean or how it applies here. Nor, in my ignorance, do I know what FormMail Special Fields are and if that would solve my problem. In every case in my forms, a request for an email address is in a unique form field (not imbedded in some larger text field).

Anyway, it sound like I need FormMail to avoid spam detection on these email form fields. Please let me know how to do that.

Thanks,
Alan

[russellr]

Hi,

Your sentence "However, your deriving of "email" from "Email" is one of our suggestions to overcome a little annoyance with fmbadhandler." is confusing and I'm not sure what you mean or how it applies here.

I mean that you have:

<input type="hidden" name="derive_fields"
  value="email=Email,realname=Name"/>

I guess you got that from our documentation.

That's required when you're using FMBadHandler.

But, it means that you're collecting an email address in the "Email" field, which is not a special field.

So, I assumed (incorrectly) that the problem is with the Contact Email part of your form (the Email field).

I've just looked at your form again, and now realize the problem is with these fields:

• q3b
• q4b
• q5b

To solve this problem, we'll have to enhance the junk detection feature along the lines I mentioned in my last post.

[aschaevitz]

OK. Now I understand. So, I believe you said you will be enhancing FormMail to handle email fields in other form fields such as my q3b field (or providing some ability to exempt those fields from spam examination). Will I get direct notification when that update is available or will it be via the update notification setting that I have in FormMail?

[russellr]

Hi,

You'll get an update notification if you've got that enabled in your current FormMail.

I'll also reply back here when it's available (provided I remember to do so).

The code is written and tested. There are a couple of other features I want to get into the next version and I have to update the documentation.

It should be less than a week away.

[russellr]

Hi,

FormMail version 1.16 is now released and contains the new feature you need.

The documentation has been updated:
http://www.tectite.com/fmdoc/attack_detection_junk.php