There are a lot of connectors in Logic Apps, and they usually make your life a lot easier but sometimes there might be even better ways to connect to an Azure Service.
The “problem”
This is not a fix-that-bug post so there really is no problem, however I think you can consider using another approach sometimes. This was evident when the team could not use the Azure Table-connector some weeks ago. Due to security reasons we had to use the HTTP-adapter and call the table storage API-directly, and in the end it solved a very big problem for us.
Azure Services APIs
A lot of Azure services have APIs. You can find documentation for them here. They include Cosmos DB, MySQL, maintenance, subscriptions and much more. If there is no connector for the thing you need to do in Azure, perhaps there is an API that you can call. Sometimes the APIs can be much more granular and have a little more finesse than the connector.
I therefore suggest you should check out the possibilities when using Logic Apps (and even functions). If you feel the connector lack a bit of refinement, or behaves in unwanted ways, take a look at the APIs.
Azure Table storage
I will use Azure Table Storage as an example. There is a Table Storage Connector that does the job, but it does not do it very well. take a look at this flow that was built using the original connector:
The original flow has been lost to time but the important thing here is to look at the remove metadata. Every call to the storage responds with three additional properties: odata.etag, PartitionKey and RowKey. We did not want to return that data to the caller and so it was removed. However, this was done using the “RemoveProperty” operation and for some strange reason the combination of that, together with the “Add to response” at the bottom every row took between 2 and 5 seconds(!). When returning rowsets of 30 rows, we where talking minutes to respond.
What can be done using the connector?
First off, you have to ask: What can I do just using the connector? In the case above, the developer could use the parameter called Select query to return only the columns needed and omitting the partitionKey and RowKey, but the adapter would still return the odata.etag, and therefore the need for one "remove metadata" and the Add to Response message would still be needed, and was the most time consuming.
Sequential vs parallel
The next thing you can look at is the flow control. In this case the data manipulation was done in a loop. Try changing the Degree of parallelism to one and run the flow again, and then try the max value. In our case it made little to no change.
Using the API directly
To start off there is and inherent problem with using the API and that is the security model and recycling of SAS-keys. You have to be aware of it, that is basically it.
Going into this part of the plan we knew we had one issue: To return only the data we needed to send back to the caller. This meant only the columns they wanted and then remove the odata.etag.
Looking at the documentation for querying tablestorage for entities we found three things to use:
Authorization
According to the documentation this was supposed to be a header but you can just as easy just use the querystring, i.e. the string you copy from the storage account to give you access
$select
The API supports the ability to ask for only a subset of the columns and thereby having the same capability as the connector.
Ask for no metadata back
By setting the Accept header to application/json;odata=nometadata you can omit any metadata.
The resulting call
Note the URL-escaping. We did not succeed in using the Queries part and I think that is due to how they are URL-encoded when sent to the service. So we had to put everything in the URI-field.
Result
By combining these we could make sure the caller would get the correct data and we did not have to manipulate it before returning the payload. This resulted in calls that responded in milliseconds instead of a full minute.