FundSvcs Community

Expand all | Collapse all

NCOA Vendors

  • 1.  NCOA Vendors

    Posted 06-17-2021 11:07 AM
    Hi, all,

    Has anyone done a comprehensive analysis of the various vendors who are certified to provide NCOA processing? From reading past posts on FundSvcs, I understand there can be variety amongst the pool of vendors with regard to their process, what they provide, and, obviously, pricing. We currently use Lorton Data but I was not involved in the contract negotiation for their services, and as we move to a new CRM this fall, I'd like to revisit our contract and determine if we should continue the relationship.

    Any recommendations of vendors welcome, but I'm particularly interested in an analysis of pros/cons and pricing.

    Thanks!
    Marie

    ------------------------------
    Marie Loson
    Director of Advancement Services
    SUNY Potsdam
    losonme@potsdam.edu
    ------------------------------


  • 2.  RE: NCOA Vendors

    Posted 06-17-2021 11:16 AM

    Hi, Marie –

     

    I can appreciate your question, as it's been one that I've raised before too.  One piece of advice some people have given me is to work with a rotation of vendors, so you can benefit from their different strengths. 

     

    I do have specific feedback about vendors we've worked with, so please feel free to email me offline if you'd like to chat further about specifics. 

     

    Thanks!

     

    Michael Halverson, Ed.D.
    Senior Director of Advancement Services
    Loyola University Chicago
    T. 312-915-7283 | C.
    320-363-4987
    mhalverson@luc.edu | www.luc.edu/advancement

     






  • 3.  RE: NCOA Vendors

    Posted 06-23-2021 09:51 AM

    I haven't done (or seen) any comprehensive investigation.  I have been part of two different narrower evaluations.

     

    One evaluation was at a previous institution, where we were implementing our NCOA process following the implementation of a new system.  We did a test comparison with two major data vendors, whose names would be familiar to you, and did a differential comparison on the results.  The marginal differences were quite small, a fraction of a percent, and not trending in favor of either vendor.

     

    More recently, we did an RFQ for a combined process that included both NCOA and deceased suppression, which had been with the same vendor for a number of years.  We didn't do an actual test of the NCOA results, but did a scoring that turned out to be primarily on cost since the vendors generally met our other requirements across the board.  The high and low costs differed by a factor of 3 or a little more, but overall cost was pretty reasonable, and one vendor's cost varied significantly if we structured our contract as an annual subscription rather than on a per-run basis, so that might be worth looking at.

     

    The NCOALINK process itself is pretty tightly regulated by the USPS.  It's worth taking a look at the detailed rules for matching names, and check out the Cindy-Mary table!

     

    The differences that I've seen tend to reflect the work done prior to the NCOALINK  process itself.  In order to compare the submitted data to the NCOA reference database, the vendor has to prepare the name to use in comparison and to validate the address so that it can be compared to the reference data.

     

    We discovered with our first vendor that, even though we submitted the name information in its component pieces (prefix, first name, middle name, etc.), their process was set up to handle other submissions that had the formatted name all in one field, and the first thing that they did was put our name all back together and then re-parse it!  How did we discover this?  Well, their name parsing had some weaknesses, esp. related to compound prefixes like "Lieutenant General" and "The Honorable" and then there were issues matching, say prefix "The", first name "Honorable," etc.  We didn't discover that until the second or third cycle when we found out that there was an exception file that we hadn't been getting of records that didn't have enough valid information to continue with the process.  That brings me to probably the most important point:  make sure that you get all the exception files from whatever vendor you use.  (Happily, we worked with the vendor and they improve their name parsing and we stopped getting these rejections.)

     

    Colleges and universities have a pretty fair advantage in these processes because, at least for students and alumni, they have pretty complete name data, whereas membership organizations may have just, say, "J. Smith", which limits match success (take a look at the Cindy-Mary table, really!).  We have ongoing work, focusing on priority constituents, to build out our name data (much of that arising from our validation reported-deceased constituents in the deceased suppression file, where the deceased person was a different relative J. Smith; that work also builds out a first name of "Mrs. John Smith" so we can match her data and not that of her-possibly deceased-husband).

     

    Address validation has similar issues.  It's likely that some addresses in your database don't include enough information to validate.  The validation process is likely to throw away parts of your address that it doesn't recognize as part of an address (addresses at colleges and universities, for instance, seem esp. prone to this!), and that may leave nothing to work with.  Similarly, your address may have the street name misspelled in a way that's unrecognizable (each vendor seems to be able to gestalt and correct some but not others), or the primary number may be missing a digit or have one too many, etc.  Here again, getting back the exception file is a good way to dig into addresses that you might think are good but aren't, and, chances are, your mailing vendor may be doing a similar process and mail pieces aren't actually being delivered.  (Of course, the converse is true; if you're digging into exceptions from your mailings, the addresses are likely to work for NCOA as well!).

     

    When we switched our NCOA vendor, we got a big jump in the number of addresses rejected as invalid, which we'd pretty much cleaned up as reported by the previous vendor, and we're triaging those and working our way through them (we're sending out 1.2+ million records, so even a fairly small percentage is still quitter a few records!).  So that was a plus, and possibly an incentive to cycle vendors from time to time.  But it's still weird.  We like our current vendor quite well, but we're finding some odd gaps-for instance, they don't seem to do very well with multi-line addresses (which, again, we discovered by looking at the exception files).

     

    I guess that I should say that we've been able to identify very few cases where NCOA has given us bad info (like, the number I can count without taking off my shoes, out of a 1.2M record process!).  Most of those legitimately match change-of-address filings, like a child with the same name that moved and didn't include enough address information to differentiate their information from their parent's!  So we flag those and suppress the changes.

     

    One other oddity that we've seen is that, when someone passes away, their surviving relative may file a change of address to the survivor's address.  We get that data from NCOA and process the change of address.  Then it messes up our deceased screening, because the deceased individual never lived at the address that we now have!

     

    And, of course, some constituents file changes of address to move between their home and their vacation home, and it's good to think about how you want to handle those.

     

    There's way more that could be said.  I did an aasp Summer Series  presentation on these sorts of issues, called "Who Cares About Addresses, Anyway?"  That might be worth a look (I hope, someday, to transform it to a Best Practice document, but don't have the time right now, and don't see that kind of  looking ahead....)

     

    The good new is that, whichever vendor you choose, you're probably in good shape, and paying attention to the exception files offers a good opportunity to identify any weaknesses, either in your vendor process or in your data.

     

    My US$0.02 worth; the usual disclaimers apply.

     

    Good luck!

     

    Alan

     

    Alan S. Hejnal   

    Data Quality Manager

     

    SNAGHTML5cbfa34

     

    Smithsonian Institution - Office of Advancement

    600 Maryland Ave SW Ste 600E

    PO Box 37012, MRC 527

    Washington, DC 20013-7012

    Voice: 202-633-8754 | Email: HejnalA@si.edu                                                                                                                                            

     






  • 4.  RE: NCOA Vendors

    Posted 06-23-2021 10:46 AM
    Highly useful, as always Alan, but you left out the links. :D


    The Cindy-Mary table ... (I had hopes of a USPS nickname table.) Is the Cindy-Mary table a first name/middle name matching scheme? I've used my own version of it, if that's what it is. 

    Also, if that's what it is, I'm not buying "Mary Cindy" matching "Cindy Mary."

    Mark







  • 5.  RE: NCOA Vendors

    Posted 06-23-2021 11:13 AM

    Thanks, Mark.

     

    Yes, the Cindy-Mary table demonstrates first name/middle name matching rules.  The most interesting part for me is that, if all you have is one initial, it's not going to match an individual move. 

     

    I'm a little more open to "Cindy Mary" matching "Mary Cindy" than you are.  Maybe not so much in higher ed, where the name data (at least for students and alumni) is pretty solid, but I've seen that sort of switch more in a membership organization context (and, looking up people in LexisNexis, I've also seen the sequencing variants reflected there as well, sometimes taking careful attention to figure out which is right, so it's not just us!).  Plus, the likelihood that at a given address and with a given last there would be both a Cindy Mary and a Mary Cindy who are different people makes the chance of a false positive seem remote.  Ish.

     

    It's probably worth noting that this all comes into play if the constituent filed an Individual move; if they filed a Family move, the last name match is enough.  It does put an interesting light on how one might choose to file a change of address.  In my household, there are three of us, with three different last names.  So do we file three family moves, even if there is only one individual with a given last name, to sweep up first name variants, or cases where someone has my spouse's first name and my last name?  Or do we file three individual moves?

     

    My US$0.02 worth; the usual disclaimers apply.

     

    Good luck!

     

    Alan

     

    Alan S. Hejnal   

    Data Quality Manager

     

    SNAGHTML5cbfa34

     

    Smithsonian Institution - Office of Advancement

    600 Maryland Ave SW Ste 600E

    PO Box 37012, MRC 527

    Washington, DC 20013-7012

    Voice: 202-633-8754 | Email: HejnalA@si.edu                                                                                                                                            

     






  • 6.  RE: NCOA Vendors

    Posted 06-23-2021 11:59 AM
    I agree that the possibility of error in an NCOA move seems quite low for the "Mary Cindy", "Cindy Mary" match. I disagree more in principle than in fact.

    Of course, I am looking at it not just in terms of NCOA, but also how we match external data to Advance when we add them. In that case, you don't restrict your results to "match"/"no match" but also allow a "review potential match" result, which is where I would place the  "Mary Cindy", "Cindy Mary" pairing. This is timely for me, because we are reviewing our own matching logic here. 

    The rules table is an interesting concept - not sure what I think. You can fudge data so many ways to find matches (Look at nicknames, calculate edit distance, a rules table, soundex/metaphone), and each in its own way has merits. However, I am not sure you can use them all for calcuational reasons, but also I'm worried that if you use nicknames, use a rules table, and then calculate edit distance, you might get "Lawrence" to match "Elizabeth.".

    I'm not entirely being facetious.

    "Lawrence" -> "Larry" -> "Lary" (nickname and rules table)
    "Elizabeth" -> "Lizy" (nickname)

    edit distance makes them close,

    Mark







  • 7.  RE: NCOA Vendors

    Posted 30 days ago

    The NCOALINK approach is interesting.  They vary one parameter at a time.  Early on in the cycle, for example, if there is no straight-up match, they "normalize" first name and try again (think Philip/Phillip, Alan/Allan/Allen/Allyn/Alun, etc.) .  If there's no match, they try again substituting a nickname for the first name for all nickname occurrences.  If there's no match, they normalize the last name. but use the input first name.  They don't vary all the elements at the same time, probably because of the sort of issues that you indicate.

     

    (I'd love to get a copy of their normalized name reference and nickname reference, though, to be fair, I've never worked very hard at it....)

     

    Sometimes even varying one parameter, like nicknames, can create bad matches.  For example, we have a couple with the first names Patrick and Patricia, and both of those reduce to "Pat", so they turned up as potential duplicates in our internal duplicate-checking until we added those records to the exclude file.

     

    I guess that's why we need review by a person. ��

     

    My US$0.02 worth; the usual disclaimers apply.

     

    Good luck!

     

    Alan

     

    Alan S. Hejnal   

    Data Quality Manager

     

    SNAGHTML5cbfa34

     

    Smithsonian Institution - Office of Advancement

    600 Maryland Ave SW Ste 600E

    PO Box 37012, MRC 527

    Washington, DC 20013-7012

    Voice: 202-633-8754 | Email: HejnalA@si.edu                                                                                                                                            

     






  • 8.  RE: NCOA Vendors

    Posted 30 days ago

    A lot of it depends on what kinds of errors we see most commonly and therefore that we need to detect during the matching process. I'd love to see an historical breakdown from an NCOA vendor on what hit rate they get by including nicknames, by normalizing first names. and finally by normalizing last names. Here at Berkeley, we also calculate a Jaro-Winkler distance and do some work with Soundex. I might throw those in also. 

    I'm guessing the final solution will be for us to build our system with an assortment of checks and measure the relative effectiveness of the different checks to see if some can be dropped.

    Mark