Wednesday, May 25, 2022

Example ElasticSearch Queries

Overview

There are three different match types in ElasticSearch

  • term - the whole field must match.  This has a limit of matching only the first 256 characters.
  • wildcard - ability to search for partial words.  Example: break* will match break and breaking.  This also has a limit of 256 characters.
  • match_phrase - This matches on a word boundary and can go longer than 256 characters.
Case insensitive searches are super slow.  Create a lowercase field in ElasticSearch instead.

Example queries that can be used for ElasticSearch and Lucene

# Count items in an index
GET customer_index/_count
{
    "query" : {
        "match_all" : {}
    }
}

# Get top 200 from the customer_index
GET customer_index/_search
{
    "size" : 200,
    "query" : {
        "match_all" : {}
    }
}

# Return the top 200 products that are movies
GET product_index/_search
{
    "size" : 200,
    "query" : {
        "bool" : {
            "must" : [
                {
                    "match" : {
                        "category" : "Movie"
                    }
                }          
            ]
        }
    }
}

# Search for any products that have DVD in the name with a wildcard
GET product_index/_search
{
    "size" : 200,
    "query" : {
        "wildcard" : {
            "name" : {
                "value" : "*DVD*"
            }
        }
    }
}

# Get an aggregate list of product categories sorted alphabetically
GET product_index/_search
{
    "size" : 0,
    "aggs" : {
        "category" : {
            "terms" : {
                "field" : "category.keyword",
                "order" : {
                    "_key" : "asc"
                }
            }
        }
    }
}

# Get a list of physical products that do not have a category
GET product_index/_search
{
    "size" : 200,
    "query" : {
        "bool" : {
            "must_not" : [
                {
                    "exists" : {
                        "field" : "category"
                    }
                }
            ],
            "minimum_should_match" : 1,
            "should" : [
                {
                    "match" : {
                        "product_type" : {
                            "query" : "Physical"
                        }
                    }
                }
            ]
        }
    }
}


# Group by Aggregate
GET _search/
{
    "size" : 0,
    "query" : {
        "bool" : {
            "must" : [
                {
                    "match" : {
                        "product_type" : "Physical"
                    }
                },
                {
                    "wildcard" : {
                        "category.keyword" : {
                            "value" : "*DVD*"
                        }
                    }
                }
               
            ]
        }
    },
    "aggs" : {
        "group_by_column" : {
            "terms" : {
                "field" : "category.keyword",
                "size" : 10000
            }
        }
   
    }
}


# Perform a search and order by using functions
GET product_index/_search
{
  "size": 200,
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "must": [
            {
              "bool": {
                "minimum_should_match": 1,
                "should": [
                  {
                    "term": {
                      "category.keyword": "Books"
                    }
                  },
                  {
                    "term": {
                      "category.keyword": "Movies"
                    }
                  }
                ]
              }
            }
          ],
          "minimum_should_match": 1,
          "should": [
            {
              "match_phrase": {
                "nameLowercase": "journey"
              }
            },
            {
              "match_phrase": {
                "descriptionLowercase": "journey"
              }
            }
          ]
        }
      },
      "functions": [
        {
          "filter": {
            "bool": {
              "must": [
                {
                  "term": {
                    "nameLowercase.keyword": "journey"
                  }
                }
              ]
            }
          },
          "weight": 3
        },
        {
          "filter": {
            "bool": {
              "must": [
                {
                  "match_phrase": {
                    "nameLowercase": "journey"
                  }
                }
              ]
            }
          },
          "weight": 2
        },
        {
          "filter": {
            "bool": {
              "must": [
                {
                  "match_phrase": {
                    "descriptionLowercase": "journey"
                  }
                }
              ]
            }
          },
          "weight": 1
        }
      ],
      "score_mode": "first",
      "boost_mode": "replace"
    }
  },
  "sort": {
    "_score": {
      "order": "desc"
    }
  }
}



Friday, April 24, 2020

Becoming an Automated Tester Using TypeScript, Selenium, NodeJS, and Mocha

Overview

In order to be an automated tester you must be able to:

  • Develop in JavaScript
  • Develop in the TypeScript language which is a super set of the JavaScript language.
  • Understand NodeJs which is a JavaScript runtime
  • Use yarn which downloads packages that other people have built in TypeScript or JavaScript.
  • Work with Selenium which is a library that interacts with the Web Browser.
  • Work with Mocha which is a testing framework.
  • Use regular expressions for matching text on a web page.


The path to being an automated tester


Learn JavaScript





Learn TypeScript


Learn NodeJS



Learn Yarn



Learn Selenium 


Learn Mocha

  • https://testautomationu.applitools.com/mocha-javascript-tests/


Learn Regular Expressions

Friday, September 6, 2019

Microsoft.Build.Utilites 2.0.0 was not able to reference the assembly Telerik.Sitefinity.OpenAccess

When running an older version of Sitefinity under Windows 10, the Telerik enhancer.exe requires the .NET Framework 3.5 to be installed.  By default Windows 10 does not come with the .NET Framework 3.5 so you will need to install it.  Windows 10 comes with the .NET Framework 4.6 installed by default.

Here is the error for future reference:
Microsoft.Build.Utilites 2.0.0 was not able to reference the assembly Telerik.Sitefinity.OpenAccess in \packages\Telerik.DataAccess.Fluent.2015.3.926.1\tools\enhancer\enhancer.exe

Friday, August 16, 2019

Deleting Logs in Salesforce

Sometimes Salesforce logs are clog up the environment and you need to delete them. 

Logs are found in two different places in Salesforce.

Deleting Apex Logs

If you receive this message, this is an indicator that you need to delete your Apex Logs

The Developer Console didn't set the DEVELOPER_LOG trace flag on your user. Having an active trace flag triggers debug logging. You have 251 MB of the maximum 250 MB of debug logs. Before you can edit trace flags, delete some debug logs.

1.  Click the gear icon and choose developer console

2.  In the query editor tab enter the query and click Execute

SELECT Id, StartTime, LogUserId, LogLength, Location FROM ApexLog


3.  Select the rows to delete and click Delete Row



Remove from Debug Logs


1.  Click on Gear Icon and then Setup


2.  Search for Logs


3.  Delete all the Logs or Just for your user




Tuesday, July 2, 2019

Creating an IIS Redirect Rule from Subdomain to the Root Domain

IIS Rewrite Rules have a lot of inherent complexity.  They use regular expressions and are difficult to debug.  The server variables have little documentation provided by Microsoft.  Recently we had an ask from the business to redirect any requests for subdomains to the main domain.  While this is simple to understand it is difficult to write.

Before you create and debug a rewrite rule you need a couple things:
1.  Install IIS by going into Programs and Features then Windows Features
2.  Install the Web Platform Installer https://www.microsoft.com/web/downloads/platform.aspx  This installs modules for IIS like NuGet installs packages for .NET
3.  Use the Web Platform Installer to install the Rewrite Module

Here is the end result:



Match URL
The thing that hung me up for most of the time was the match url.  This is not the full URL as you would expect but simply the page that was requested like default.aspx.  This is Mistake #1 from Lex Li:  https://blog.lextudio.com/the-very-common-mistakes-when-using-iis-url-rewrite-module-a2ab7e4fee59

Condition Order
In order to do back references to conditions (see the {C:2} and {C:3} variables) you have to put the negate false conditions first.  If you don't the back reference does not work.

HTTP_HOST Matching
If you do not put the regular expression carat ^ for start and the dollar $ sign for end, IIS will redirect until the URL is too long and it throws an error.

Putting It All Together
https://www. - We always want to redirect to secure www
{C:2} and {C:3} - This is a back reference to the last condition, the last set of parenthesis.
{R:1} - This is the requested page from the Match URL

Debugging the Redirect Rule

1.  Here is some excellent documentation how to create a simple test file to debug your rewrite rule:

2.  I used Expresso to debug the regular expression to match the subdomain.rootdomain.  You can also use IIS by double clicking on the Re-write module for the site.    http://www.ultrapico.com/expresso.htm  

3.  What I found to be very useful was to change the redirect url to a query string to see what the variables were.  
<action type="Redirect" url="http://www.somesitethatdoesnotexist.com/?{C:2}-{R:1}" appendQueryString="false" />


Here is the reference documentation from Microsoft:
https://docs.microsoft.com/en-us/iis/extensions/url-rewrite-module/url-rewrite-module-20-configuration-reference





Friday, June 21, 2019

Resetting the Admin Password in Sitefinity

If you lost the admin password for a site you can reset the password for the admin user in the database.  This will set the password format to clear text with no salt for the encryption.

UPDATE sf_users
SET    passwd = 'password',
       salt = NULL,
       password_format = 0
WHERE  user_name = 'someadminuser' 

Tuesday, May 21, 2019

How to redirect from HTTP to HTTPS in a web.config in CrownPeak CMS


<rewrite>
  <rules>
  <%
  if (context.PublishingPackage.PackageName.Contains("Live")
    {
    %>
    <rule name="Redirect to https" stopProcessing="true">
      <match url="(.*)" />
      <conditions>
        <add input="{HTTPS}" pattern="off" ignoreCase="true" />
      </conditions>
      <action type="Redirect" url="https://{HTTP_HOST}{REQUEST_URI}" redirectType="Permanent" appendQueryString="false" />
    </rule>
    <%
  }
  %>
  </rules>
</rewrite>