Xano Transform Engine

What is Xano Transform?

The Xano Transform Engine is a new technology built by Xano that takes the traditional dot notation syntax and upgrades it to support data manipulation. This syntax is friendly to objects and arrays and supports conditional filtering using the new Expression data type.

Xano Transform encompasses some updates to our GET and SET filters, allowing you to use a powerful and expansive new type of filtering syntax to parse, return, and transform your data in exactly the way you need. Consider it an alternative to chaining several filters and functions together to accomplish a specific result.


Why Xano Transform?

Xano is, and will always remain, a No-Code first platform. That doesn't mean that we don't want to offer you more ways to accomplish your goals in the fastest way possible. While this may feel like a leap to "traditional development" methods, at its core, Xano Transform is here to provide both traditional developers a more familiar way to work with their data, and empower the no-code audience to work faster with a specific set of easy to learn syntax.

Xano Transform also offers you an easy way to share solutions with other Xano users by just copying and pasting an expression to them. This is an incredibly helpful tool while we work behind the scenes to offer other methods of function stack portability.


How do I use Xano Transform?

Xano Transform has two different operating 'modes', as an extension of our GET filter, and a separate expression data type, which all work from utilizing a standardized syntax, outlined in this documentation.

Please note that you can use standard mathematical operators to compare values in areas where they would normally be accepted, such as >, <, ==, !=, etc...


The $$ special variable

Throughout this documentation, there will be repeated mentions of the $$ special variable. This is our version of the standard this variable, which is commonly used in various programming languages and higher order filters. This concept is used to represent the context that is being referenced within a function/filter.

In this example, $$ represents the expression1+2+3

(1+2+3)|transform:$$+1                   // 7

One common problem with the $$/this variable concept, is what happens when you need to reference 2 or more different versions of it?

(1+2+3)|transform:$$+(1|transform:$$+1)

What if I wanted to reference the first $$ in the second transform? There isn't a way to do this because the existence of the second transform has taken over the previous value of the $$ variable. The solution tends to involve manually creating new variables and assigning them to different copies of the $$ variable.

Xano gets around this by having $$ represent more than just a value - it represents the top most value of anything binded to it.

All $$ variables are available via the following syntax. $0, $1, $2 - or $[0], $[1], $[2], etc. This number keeps increasing as values are bounded to it. This means that the above example could also be written as the following:

(1+2+3)|transform:$0+(1|transform:$1+1)

Therefore, if we needed to access the first $$ within the 2nd transform, we could do so by referencing $[0].

(1+2+3)|transform:$$+(1|transform:$$+$0)

Sometimes, it is easier to reference items from the top instead of the bottom. Since Xano supports negative indexes, the second item from the top is represented as -2. Therefore, we could also use the following syntax using the array notation:

(1+2+3)|transform:$$+(1|transform:$$+$[-2])

Retrieval with GET

Example Data Payload

{
  "v": 10,
  "items": [
    {
      "id": 1,
      "name": "s1",
      "score": 100,
      "tags": [
         {"tag":"beta","weight":5},
	 {"tag":"test","weight":50}
      ]
    },
    {
      "id": 2,
      "name": "x9",
      "score": 87,
      "tags": [
        {"tag":"abc","weight":10},
	{"tag":"def","weight":90}
      ]
    }
  ]
}
SyntaxResult

v

10

items

[
    {
      "id": 1,
      "name": "s1",
      "score": 100,
      "tags": [
         {"tag":"beta","weight":5},
	 {"tag":"test","weight":50}
      ]
    },
    {
      "id": 2,
      "name": "x9",
      "score": 87,
      "tags": [
        {"tag":"abc","weight":10},
	{"tag":"def","weight":90}
      ]
    }
  ]

items[0]

{
  "id": 1,
  "name": "s1",
  "score": 100,
  "tags": [
    {"tag":"beta","weight":5},
    {"tag":"test","weight":50}
  ]
}

items[0].id

1

items[0].tags[0]

{"tag":"beta","weight":5}

items[0].tags[0].tag

"beta"

Array Retrieval

SyntaxResult

items.id

[1,2]

items.tags

[
  {"tag":"beta","weight":5},
  {"tag":"test","weight":50},
  {"tag":"abc","weight":10},
  {"tag":"def","weight":90}
]

items.tags.tag

["beta","test","abc","def"]

Conditional Retrieval

Conditional retrieval is made possible by using the expression syntax within the array brackets of an item.

The $$ special variable is used to represent the current context of the iterated item of the array.

As seen below, this syntax is supported across multiple levels for deeply nested arrays.

SyntaxResult

items[$$.id > 1]

[{
  "id": 2,
  "name": "x9",
  "score": 87,
  "tags": [
    {"tag":"abc","weight":10},
    {"tag":"def","weight":90}
  ]
}]

items[$$.id > 1].id

[2]

items[$$.id > 1].tags.tag

["abc","def"]

items[$$.id > 1].tags[$$.weight>=90].tag

["def"]

Automatic Anchoring

Sometimes you may need to reference something outside of the $$ special variable. For example, you may want to conditionally select something based on a value that is located in the original root of the object.

This type of reference is supported through automatic anchoring. Each item element of the breadcrumb hierarchy is numerically indexed through $0, $1, $2, $N variables.

Example: items[$$.id > 1].tags[$$.weight>=90].tag

$2 and $4 have multiple values since they represent an array iteration. The example below is just showing the snapshot of one iteration of their value.

Special VariableResult

$0

{
  "v": 10,
  "items": [
    {
      "id": 1,
      "name": "s1",
      "score": 100,
      "tags": [
         {"tag":"beta","weight":5},
	 {"tag":"test","weight":50}
      ]
    },
    {
      "id": 2,
      "name": "x9",
      "score": 87,
      "tags": [
        {"tag":"abc","weight":10},
	{"tag":"def","weight":90}
      ]
    }
  ]
}

$1

[
  {
    "id": 1,
    "name": "s1",
    "score": 100,
    "tags": [
      {"tag":"beta","weight":5},
      {"tag":"test","weight":50}
    ]
  },
  {
    "id": 2,
    "name": "x9",
    "score": 87,
    "tags": [
      {"tag":"abc","weight":10},
      {"tag":"def","weight":90}
    ]
  }
]

$2

{
  "id": 2,
  "name": "x9",
  "score": 87,
  "tags": [
    {"tag":"abc","weight":10},
    {"tag":"def","weight":90}
  ]
}

$3

[
  {"tag":"abc","weight":10},,
  {"tag":"def","weight":90}
]

$4

{"tag":"def","weight":90}

$5

"def"

The $$ special variable is available to be used as a convenience. It always represents the top most numerically indexed special variable that is available within its context.

SyntaxResult

items[$$.id > 1].tags[$$.weight>=90].tag

["def"]

items[$2.id > 1].tags[$$.weight>=90].tag

["def"]

items[$$.id > 1].tags[$4.weight>=90].tag

["def"]

items[$2.id > 1].tags[$4.weight>=90].tag

["def"]

Advanced Examples

SyntaxResult

items[$$.id > 1].tags[$$.weight>=$0.v].tag

["abc","def"]

items[$$.id > ([$0.v,2,3]|max)].id

[]

items[$$.id > ([$0.v,2,3]|min) || $$.id == 1].id

[1,3]

Transformation with SET

In addition to the syntax above for data retrieval, SET supports new syntax for data transformation.

The SET filter leverages the targeting mechanism of the GET filter and allows you to transform the data that is retrieved. It is important to note that after the transformation, the complete object (not just what got changed) is returned. By doing so, it allows for this process to be repeated as many times as you need to complete your transformations.

Below are some examples combining the above syntax with the SET filter.

Multiple SET filters may be required for each transformation.

SyntaxResult

Change the value of "v"

Path: v Value: 50

{"v": 50,...

Change the value of all IDs in the items

Path: items.id Value: 99

{
  "v": 10,
  "items": [
    {
      "id": 99,
      "name": "s1",
      "score": 100,
      "tags": [
         {"tag":"beta","weight":5},
	 {"tag":"test","weight":50}
      ]
    },
    {
      "id": 99,
      "name": "x9",
      "score": 87,
      "tags": [
        {"tag":"abc","weight":10},
	{"tag":"def","weight":90}
      ]
    }
  ]
}

Add 1 to all item IDs

Path: items.id Value: $$+1

{
  "v": 10,
  "items": [
    {
      "id": 2,
      "name": "s1",
      "score": 100,
      "tags": [
         {"tag":"beta","weight":5},
	 {"tag":"test","weight":50}
      ]
    },
    {
      "id": 3,
      "name": "x9",
      "score": 87,
      "tags": [
        {"tag":"abc","weight":10},
	{"tag":"def","weight":90}
      ]
    }
  ]
}

Uppercase all item tags

Path: items.tags.tag Value: $$|to_upper

{
  "v": 10,
  "items": [
    {
      "id": 1,
      "name": "S1",
      "score": 100,
      "tags": [
         {"tag":"BETA","weight":5},
	 {"tag":"TEST","weight":50}
      ]
    },
    {
      "id": 2,
      "name": "X9",
      "score": 87,
      "tags": [
        {"tag":"ABC","weight":10},
	{"tag":"DEF","weight":90}
      ]
    }
  ]
}

Add the value of an input to each tag weight

Path: items.tags.weight Value: $input.add+$$ Input: 500

{
  "v": 10,
  "items": [
    {
      "id": 1,
      "name": "S1",
      "score": 100,
      "tags": [
         {"tag":"BETA","weight":505},
	 {"tag":"TEST","weight":550}
      ]
    },
    {
      "id": 2,
      "name": "X9",
      "score": 87,
      "tags": [
        {"tag":"ABC","weight":510},
	{"tag":"DEF","weight":590}
      ]
    }
  ]
}

Add a new total_weight key for each item that contains the sum of all tag weights

Path: items.total_weight Value: $1.tags.weight|sum

{
  "v": 10,
  "items": [
    {
      "id": 1,
      "name": "S1",
      "score": 100,
      "tags": [
         {"tag":"BETA","weight":5},
	 {"tag":"TEST","weight":50}
      ],
      "total_weight": 55
    },
    {
      "id": 2,
      "name": "X9",
      "score": 87,
      "tags": [
        {"tag":"ABC","weight":10},
	{"tag":"DEF","weight":90}
      ],
      "total_weight": 100
    }
  ]
}

Expression Data Type

The expression data type can be applied to any value input and allows you to perform certain operations on your data when establishing new values. You can also reference other variables and inputs in your expressions which allows for faster data manipulation.

Please see our full, in-depth documentation on the Expression data type here.


Expression Filters

The Xano Transform Engine provides several filters to assist in making transformation easier using the expression syntax. It also includes custom implementations of the higher order filters.

as

The as filter provides a way to dynamically assign a variable to an expression while chaining together filters. This is useful for long or dynamic expressions that need to be referenced more than once.

Arguments

  • name - the new variable name

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

Examples

// initial data
$obj.really.long.ref.company             // foobar, inc https://foo.bar

// without as
$obj.really.long.ref.company|substr:0:($obj.really.long.ref.company|index:http)|trim
// foobar, inc

// with as
$obj.really.long.ref.company|as:C|substr:0:($C|index:http)|trim
// foobar, inc

transform

This filter is similar to the map filter, except it can bind to all data - not just an array.

The transform filter is universal way to transform data. It works with arrays, objects, and scalar values.

Arguments

  • expression - the expression being used to transform $$.

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

Examples

[1,2,3]|transform:($$|count)          // 3
[1,2,3]|transform:($$|count)+($$|sum) // 9

{first:Alpha,last:Beta}|transform:$$.first~" "~$$.last
// Alpha Beta

to_expr

It is very important to sanitize your data, if you are using this with user generated expressions. This filter provides a way to dynamically reference your variables and use the filters within Xano.

The to_expr filter is introduces the possibility of dynamic programming. It converts text into an expression.

Arguments (none)

Context Variables (none)

Examples

"1+2+3"|to_expr                      // 6

$input.score = 10
"1+2+3+$input.score"|to_expr         // 16

get

See above for a detailed walkthrough of the improvements to the get filter.

set

See above for a detailed walkthrough of the improvements to the set filter.

Higher Order Filters

These filters resemble their Lambda counterparts, but if used within an Expression, they have a simplified syntax along with their own high performance implementation. All of these have the requirement of binding to an array. There is also a subtle change where the Lambda $this variable is represented as the Expression $$ variable.

map

This filter binds to an array. If you need to transform an object or scalar value, try the transform filter.

The map filter is used to do transformations on array elements.

Arguments

  • expression - the expression being applied to each iteration of the array

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

  • $index - the context variable that represents the numerical index of the element of the array being processed

  • $parent - the context variable that represents the entire array with all of its elements.

Examples

[1,2,3]|map:$$+1                      // [2,3,4]
[1,2,3]|map:$$+1+$index               // [2,4,6]
[1,2,3]|map:$$+1+($parent|count)      // [5,6,7]

[{name:foo},{name:bar}]|map:$$.name   // [foo,bar]

some

The some filter (also called has or has any element) returns true, if the conditional expression matches any of the array elements - otherwise, false is returned.

Arguments

  • expression - the expression being applied to each iteration of the array

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

  • $index - the context variable that represents the numerical index of the element of the array being processed.

  • $parent - the context variable that represents the entire array with all of its elements.

Examples

[1,2,3]|some:$$==2                    // true
[1,2,3]|some:$$>10                    // false

every

The every filter (also called find every element) returns true if the conditional expression matches all of the array elements - otherwise, false is returned.

Arguments

  • expression - the expression being applied to each iteration of the array

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

  • $index - the context variable that represents the numerical index of the element of the array being processed.

  • $parent - the context variable that represents the entire array with all of its elements.

Examples

[1,2,3]|every:$$>=1 && $$<=3          // true
[1,2,3]|every:$$>10                   // false

find

The find filter (also called find first element) returns the array item, that first matches the conditional expression - otherwise, null is returned.

Arguments

  • expression - the expression being applied to each iteration of the array

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

  • $index - the context variable that represents the numerical index of the element of the array being processed.

  • $parent - the context variable that represents the entire array with all of its elements.

Examples

[1,2,3]|find:$$==2                    // 2
[1,2,3]|find:$$>1                     // 2

[{name:foo},{name:bar}]|find:($$.name|starts_with:b)
// {name:bar}

findIndex

The findIndex filter (also called find first element index) returns the index of the array item, that first matches the conditional expression - otherwise, null is returned.

Arguments

  • expression - the expression being applied to each iteration of the array

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

  • $index - the context variable that represents the numerical index of the element of the array being processed.

  • $parent - the context variable that represents the entire array with all of its elements.

Examples

[1,2,3]|findIndex:$$==2               // 1
[1,2,3]|findIndex:$$>1                // 1

[{name:foo},{name:bar}]|findIndex:($$.name|starts_with:b)
// 1

filter

The filter filter (also called find all elements) returns a new array of items that match the conditional expression.

Arguments

  • expression - the expression being applied to each iteration of the array

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

  • $index - the context variable that represents the numerical index of the element of the array being processed.

  • $parent - the context variable that represents the entire array with all of its elements.

Examples

[1,2,3]|filter:$$==2                  // [2]
[1,2,3]|filter:$$>1                   // [2,3]

reduce

This also has an optional 2nd argument that represents the initial value. If not specified, it defaults to 0.

The reduce filter is a bit more complicated than the other because it has state that changes through each iteration of the array elements. The purpose of this filter is to perform some calculation based on all of the array items. A common example would be summing a list of numbers together one after another.

Arguments

  • expression - the expression being applied to each iteration of the array

  • initialValue - (optional) the initial value for $result. This defaults to 0, if not specified.

Context Variables

  • $$ - the context variable that represents the element of the array being processed.

  • $index - the context variable that represents the numerical index of the element of the array being processed.

  • $parent - the context variable that represents the entire array with all of its elements.

  • $result - the context variable that is used to represent the result of the previous iteration or the initial value on the first iteration.

Examples

[1,2,3]|reduce:$$+$result             // 6
[1,2,3]|reduce:$$+$result:10          // 16

Use Cases & Examples

Xano Transform is almost limitless. Some of the most common use cases for Transform are as follows. Please note that this is not an extensive list, and we recommend reading on for more information and examples.

For the sake of these examples, assume we are working with the following data set. You can download the full list of products as CSV or JSON below as well if you want to try these yourself.

{
    result:
            [
                {
                    id: 1,
                    created_at: 1702311936290,
                    title: iPhone 9,
                    description: An apple mobile which is nothing like apple,
                    price: 549,
                    discountPercentage: 12.96,
                    rating: 4.69,
                    stock: 94,
                    brand: Apple,
                    category: smartphones
                },
                {
                    id: 2,
                    created_at: 1702311936290,
                    title: iPhone X,
                    description: SIM-Free, Model A19211 6.5-inch Super Retina HD display with OLED technology A12 Bionic chip with ...,
                    price: 899,
                    discountPercentage: 17.94,
                    rating: 4.44,
                    stock: 34,
                    brand: Apple,
                    category: smartphones
                },
            ...]
}

Using Xano Transform, we could use one or more of the following:

  • Find me all of the products with a price greater than $700

  • Find me all of the smartphones that have a rating of at least 4.5 and are less than $300

  • Find me all of the iPhones that have a total price of their returned price + 7.5% to account for tax

These are just a few examples of what Xano Transform could accomplish.

Find all products with a price greater than $700

result[$$.price>700]
  • result (Find all products)

    • Selects something in the JSON, in this case the 'list' key

  • []

    • Indicates that we want to apply a condition to the selection

  • $$.price (with a price)

    • Use the value of 'price' as part of the expression

  • >700 (greater than 700)

    • The expression used to define the data we want returned

Find all products with a price greater than $700 and a rating greater than 4.5

result[$$.price>700 && $$.rating>4.5]
  • result (Find all products)

    • Selects something in the JSON, in this case the 'list' key

  • []

    • Indicates that we want to apply a condition to the selection

  • $$.price (with a price)

    • Use the value of 'price' as part of the expression

  • >700 (greater than 700)

    • The first part of the expression used to define the data we want returned

  • && (and)

    • Indicate that this expression contains multiple conditions

  • $$.rating (a rating)

    • Select the 'rating' value as a part of this condition

  • >4.5 (greater than 4.5)

    • The second part of the expression used to define the data we want returned

Find all products with a price greater than $700 and a rating greater than 4.5, that are in stock

result[$$.price>700 && $$.rating>4.5 && $$.stock>0]
  • result (Find all products)

    • Selects something in the JSON, in this case the 'list' key

  • []

    • Indicates that we want to apply a condition to the selection

  • $$.price (with a price)

    • Use the value of 'price' as part of the expression

  • >700 (greater than 700)

    • The first part of the expression used to define the data we want returned

  • && (and)

    • Indicate that this expression contains multiple conditions

  • $$.rating (a rating)

    • Select the 'rating' value as a part of this condition

  • >4.5 (greater than 4.5)

    • The second part of the expression used to define the data we want returned

  • && (and)

    • Indicate that this expression contains multiple conditions

  • $$.stock>0 (that are currently in stock)

    • The third part of the expression used to define the data we want returned

Take the returned list of products and capitalize all product names

Path: name
Value: $$|to_upper
  • Path: name

    • Set the 'path' option to name to indicate this is the key to update

  • Value: $$|to_upper

    • $$

      • Select the current name

    • |to_upper

      • Apply the to_upper filter

Take the returned list of products, capitalize all product names, and conditionally update the price based on the stock value while applying proper price formatting

To complete this example, you'll want to apply the SET filter for name as described in the previous example.

Path: price
Value: $1.stock>50 ? ("$"~($$*0.9|number_format:2:.:,)) : ("$"~($$*1.3|number_format:2:.:,))
  • Path: price

    • Set the path option to 'price' to indicate this is the key to update

  • Value: $1.stock>50 ? ("$"~($$*0.9|number_format:2:.:,)) : ("$"~($$*1.3|number_format:2:.:,))

    • $1.stock>50

      • Use anchored selection to get the stock count and check to see if it is greater than 50

    • ?

      • Apply a condition based on what the previous portion of the expression returns

    • If the stock > 50

      • ("$"~

        • Begin the value with a $ character

      • ($$*0.9

        • Multiply the price by 0.9

      • |number_format:2:.:,))

        • Apply the number_format filter to format the number as a standard price (2 decimal points, a . decimal separator, and a , thousands seperator)

    • If the stock < 50

      • ("$"~

        • Begin the value with a $ character

      • ($$*1.3

        • Multiply the price by 1.3

      • |number_format:2:.:,))

        • Apply the number_format filter to format the number as a standard price (2 decimal points, a . decimal separator, and a , thousands seperator)

Troubleshooting

Text vs Expression

The get and set filters parse the path argument using the Xano expression syntax. If you have a path that looks like an expression, but needs to be interpreted as text, then wrap it with brackets and quotes.

Some 3rd party APIs (like Stripe) rely on dot notation arguments being sent as text and then interpreted on their side. In this scenario, it is important to wrap them with the bracket syntax below using double quotation marks.

The Import cURL feature has been updated to properly escape all arguments.

AS ARRAYAS TEXT

expand[0]

["expand[0]"]

line_item[0][data]

["line_item[0][data]"]

Last updated