<rss xmlns:a10="http://www.w3.org/2005/Atom" version="2.0"><channel xml:base="https://zhenyuan.azurewebsites.net/"><title>ZHEN YUAN</title><description>Latest blog posts</description><a10:link href="https://zhenyuan.azurewebsites.net/" /><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/simplifying-api-development:-build-graphql-apis-effortlessly-with-microsoft-fabric</guid><link>https://zhenyuan.azurewebsites.net/post/simplifying-api-development:-build-graphql-apis-effortlessly-with-microsoft-fabric</link><a10:author><a10:name /></a10:author><title>Simplifying API Development: Build GraphQL APIs Effortlessly with Microsoft Fabric</title><description>&lt;p&gt;In today’s fast-paced digital world, delivering data to applications quickly, securely, and flexibly is crucial. That's why developers love GraphQL — it gives them the power to query exactly what they need, nothing more, nothing less. But setting up and maintaining a GraphQL API from scratch can be time-consuming.&lt;p&gt;Enter &lt;strong&gt;Microsoft Fabric&lt;/strong&gt;: your all-in-one data platform that now makes it incredibly easy to expose your data as a &lt;strong&gt;GraphQL API&lt;/strong&gt;, with minimal setup and zero infrastructure overhead.&lt;p&gt;&lt;br&gt;&lt;h4&gt;Why GraphQL in Microsoft Fabric?&lt;/h4&gt;&lt;p&gt;Traditionally, building APIs meant provisioning infrastructure, writing boilerplate code, and managing endpoints. But Fabric changes the game by:&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Auto-generating GraphQL APIs&lt;/strong&gt; directly over your Lakehouse tables&lt;/p&gt;&lt;li&gt;&lt;p&gt;Providing a &lt;strong&gt;no-code or low-code experience&lt;/strong&gt; for developers and data engineers&lt;/p&gt;&lt;li&gt;&lt;p&gt;Offering a &lt;strong&gt;single, unified API endpoint&lt;/strong&gt; to query multiple data sources within your workspace&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;This means you can go from raw data to a production-ready API in minutes.&lt;h5&gt;Here's how easy it is:&lt;/h5&gt;&lt;ol&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Create a Semantic Model&lt;/strong&gt; in Microsoft Fabric using your Lakehouse data.&lt;/p&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Enable API Access&lt;/strong&gt; to the model via the &lt;strong&gt;GraphQL API endpoint&lt;/strong&gt;.&lt;/p&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Query your data&lt;/strong&gt; with standard GraphQL syntax — no backend service, no custom resolvers required.&lt;/p&gt;&lt;/li&gt;&lt;/ol&gt;&lt;h5&gt;Example GraphQL Query:&lt;br&gt;&lt;br&gt;&lt;/h5&gt;&lt;p&gt;query {&lt;br&gt;&amp;nbsp;&amp;nbsp; SalesOrders {&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; OrderID&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Customer {&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Name&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Email&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; }&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; TotalAmount&lt;br&gt;&amp;nbsp;&amp;nbsp; }&lt;br&gt;
}&lt;br&gt;Here is an example for GraphQL API in fabric. You can then copy the endpoint in your application and start querying the data.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/ea542cf9-9258-4c25-b9ed-e6e52d363a9d.png"&gt;&lt;img width="725" height="367" title="1" style="display: inline; background-image: none;" alt="1" src="https://zhenyuan.azurewebsites.net/posts/files/95a8fd72-3fee-4eaf-b5c1-321d2bdbae63.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;</description><pubDate>Thu, 26 Jun 2025 02:01:02 Z</pubDate><a10:updated>2025-06-26T02:01:02Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/level-up-your-agents:-deep-dive-into-topics,-orchestration,-and-actions-with-copilot-studio</guid><link>https://zhenyuan.azurewebsites.net/post/level-up-your-agents:-deep-dive-into-topics,-orchestration,-and-actions-with-copilot-studio</link><a10:author><a10:name /></a10:author><title>Level Up Your Agents: Deep Dive into Topics, Orchestration, and Actions with Copilot Studio</title><description>&lt;p&gt;&lt;img alt="Power Virtual Agents sind jetzt Teil von Microsoft Copilot Studio ..." src="https://blog.topedia.com/wp-content/uploads/2023/11/MicrosoftCopilotStudio_1200x627-1.png"&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;What is Microsoft Copilot Studio?&lt;/strong&gt;&lt;/p&gt;&lt;p align="justify"&gt;Microsoft Copilot Studio is a powerful, user-friendly environment designed to streamline the development, management, and deployment of intelligent conversational AI agents. Part of Microsoft's broader Copilot ecosystem, Copilot Studio allows users to effortlessly define &lt;strong&gt;topics&lt;/strong&gt; that guide conversations, implement precise &lt;strong&gt;orchestration&lt;/strong&gt; to ensure dialogues flow logically and contextually, and create customizable &lt;strong&gt;actions&lt;/strong&gt; to seamlessly integrate with backend services and APIs. Built to empower both developers and non-technical users, Copilot Studio simplifies the creation of sophisticated, personalized AI-driven experiences, significantly accelerating time-to-value while maintaining flexibility and control.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p align="justify"&gt;In this blog post, I'll dive deeper into how we can effectively leverage &lt;strong&gt;topics&lt;/strong&gt;, &lt;strong&gt;orchestration&lt;/strong&gt;, and &lt;strong&gt;actions&lt;/strong&gt; in Microsoft Copilot Studio to build sophisticated and dynamic conversational agents. I'll first explore how clearly defined &lt;strong&gt;topics&lt;/strong&gt; help your agent better understand user intent, then explain how &lt;strong&gt;orchestration&lt;/strong&gt; enables smooth conversational flows by managing context and transitions between different interactions. Finally, I'll show how incorporating custom &lt;strong&gt;actions&lt;/strong&gt; can extend the agent's capabilities, allowing seamless integration with external services and providing richer, more personalized experiences.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;What is a topic?&lt;/strong&gt;&lt;/p&gt;&lt;p align="justify"&gt;A &lt;strong&gt;topic&lt;/strong&gt; in Microsoft Copilot Studio represents a specific area or scenario of conversation that your AI agent can recognize and respond to effectively. Think of topics as conversational building blocks, each designed to handle particular user intents or questions. For example, you might have topics around booking appointments, answering product FAQs, or troubleshooting common issues. Defining clear and targeted topics helps your agent quickly detect what the user wants, allowing it to deliver focused and accurate responses, resulting in more natural and satisfying interactions.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/2839c6df-7b75-449f-9ce9-d7ee9f2d042c.gif"&gt;&lt;img width="730" height="411" title="topics" style="display: inline;" alt="topics" src="https://zhenyuan.azurewebsites.net/posts/files/a36f2356-40d4-45da-8ad0-62b688e3862f.gif"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;What is orchestration?&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Orchestration&lt;/strong&gt; in Microsoft Copilot Studio is the process of intelligently managing and guiding conversational flows across different topics and actions to ensure smooth, logical, and context-aware interactions. Think of orchestration as the conversation conductor, seamlessly deciding when and how to transition between different topics, invoking the appropriate actions, and maintaining the context throughout the dialogue. Good orchestration ensures that your AI agent can handle complex user journeys, adapt dynamically to user inputs, and deliver coherent, engaging, and human-like conversational experiences.&amp;nbsp; &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;As illustrated in the example above, orchestration makes decisions on which topic to navigate to next, based on the current context of the conversation. Effective orchestration ensures that your AI agent can handle complex user journeys, adapt dynamically to user inputs, and deliver coherent, engaging, and human-like conversational experiences.&lt;/p&gt;&lt;p&gt;&lt;img alt="A visual representation of conversational orchestration in AI agents. Illustrate a central node labeled 'Orchestration' connecting dynamically to multiple conversational topics and actions, represented as circles around it, with arrows showing smooth transitions between them. Use a modern, clean design, with bright colors on a white background, to convey clarity and intelligence." src="https://files.oaiusercontent.com/file-KEDuFrfuhLWSRAJFrRFXJD?se=2025-03-25T00%3A20%3A36Z&amp;amp;sp=r&amp;amp;sv=2024-08-04&amp;amp;sr=b&amp;amp;rscc=max-age%3D604800%2C%20immutable%2C%20private&amp;amp;rscd=attachment%3B%20filename%3D77b053c9-95e8-4ce5-99b1-ebef960d2fa0.webp&amp;amp;sig=EnHlLHnJsDIsuPIdpNKhKRvhhdiGCa074RaHH/D4/xI%3D"&gt;&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/b24da108-3391-4ec4-94cd-5ae3c27d39f8.jpg"&gt;&lt;img width="730" height="417" title="orchestration" style="display: inline; background-image: none;" alt="orchestration" src="https://zhenyuan.azurewebsites.net/posts/files/1bf2667b-5e04-45e9-821b-7646fa294f13.jpg" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;br&gt;&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;W&lt;strong&gt;hat is action?&lt;/strong&gt;&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;An &lt;strong&gt;action&lt;/strong&gt; in Microsoft Copilot Studio is a powerful capability that allows your conversational agent to interact with external systems, APIs, or backend processes to execute tasks or retrieve dynamic information. Actions extend the functionality of your AI agent beyond static responses, enabling it to perform real-world operations like checking inventory, scheduling meetings, processing orders, or pulling up personalized user data. By integrating actions, your conversational experiences become more meaningful, relevant, and capable of addressing users' real-time needs directly within the conversation.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/ea70e487-bb01-449a-b6aa-27075ecb6dad.gif"&gt;&lt;img title="actions" style="display: inline;" alt="actions" src="https://zhenyuan.azurewebsites.net/posts/files/f24e2861-344c-413d-bc8c-f0c2c737acae.gif"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;Now that we've explored the concepts of &lt;strong&gt;topics&lt;/strong&gt;, &lt;strong&gt;orchestration&lt;/strong&gt;, and &lt;strong&gt;actions&lt;/strong&gt; in Microsoft Copilot Studio, let's dive in together to see how you can apply them practically and elevate your bot conversations to the next level.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;In this demo, I'll showcase a practical use case: building a Copilot agent designed to keep my team updated with the latest azure updates. &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Step one:&lt;/strong&gt; Create copilot in Microsoft Copilot Studio&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/c2171698-446b-4932-b75c-d9c3c3578837.png"&gt;&lt;img width="730" height="431" title="screen1" style="display: inline; background-image: none;" alt="screen1" src="https://zhenyuan.azurewebsites.net/posts/files/c503e6a7-cdc0-4e09-bdeb-9de2d4aea9b9.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Step Two:&lt;/strong&gt; Create topic&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/8361b189-4ace-4da9-b022-895b2b7809c7.png"&gt;&lt;img width="730" height="228" title="screen2" style="display: inline; background-image: none;" alt="screen2" src="https://zhenyuan.azurewebsites.net/posts/files/b8204a79-d1a4-407d-a8da-480270811b91.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Step Three: &lt;/strong&gt;Enable AI Orchestration&lt;/p&gt;&lt;p&gt;&amp;nbsp;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/41e8a1dd-405a-4efd-9f2c-122d60e59e17.png"&gt;&lt;img width="730" height="88" title="screen3" style="display: inline; background-image: none;" alt="screen3" src="https://zhenyuan.azurewebsites.net/posts/files/181531b5-eff2-40ad-842c-ffda442120f4.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Step Four&lt;/strong&gt;: Create action to integrate with Teams.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/a5c445e9-b934-454c-964c-d23d7a7d457b.png"&gt;&lt;img width="730" height="519" title="screen4" style="display: inline; background-image: none;" alt="screen4" src="https://zhenyuan.azurewebsites.net/posts/files/5a977245-b225-4afa-bb6f-ebb410998594.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;Let’s check it out the full demo here.&lt;/p&gt;&lt;p&gt;&lt;a title="https://youtu.be/b3axMOtt8yk?feature=shared" href="https://youtu.be/b3axMOtt8yk?feature=shared"&gt;https://youtu.be/b3axMOtt8yk?feature=shared&lt;/a&gt;&lt;/p&gt;&lt;p&gt;Once you’ve done the above, you will receive a notification in Teams&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&lt;p align="justify"&gt;In conclusion, harnessing the power of &lt;strong&gt;topics&lt;/strong&gt;, &lt;strong&gt;orchestration&lt;/strong&gt;, and &lt;strong&gt;actions&lt;/strong&gt; in Microsoft Copilot Studio allows you to create sophisticated, context-aware, and highly interactive conversational agents. By strategically defining clear conversational paths with topics, smoothly managing the dialogue flow with intelligent orchestration, and integrating practical actions for real-world functionality, you can significantly enhance user experience and boost productivity. Now, equipped with these insights, you're ready to build smarter, more dynamic AI-driven interactions and elevate your agent to the next level. Happy building!&lt;/p&gt;</description><pubDate>Tue, 25 Mar 2025 03:59:33 Z</pubDate><a10:updated>2025-03-25T03:59:33Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/how-to-set-up-a-data-pipeline-in-azure-data-factory-for-importing-data-from-azure-blob-storage-to-databricks-delta-lake</guid><link>https://zhenyuan.azurewebsites.net/post/how-to-set-up-a-data-pipeline-in-azure-data-factory-for-importing-data-from-azure-blob-storage-to-databricks-delta-lake</link><a10:author><a10:name /></a10:author><title>How to Set Up a Data Pipeline in Azure Data Factory for Importing Data from Azure Blob Storage to Databricks delta lake</title><description>&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/743c6e8d-f107-4152-beb0-ca9811d46043.png"&gt;&lt;img title="Import data from Azure blob storage into Databricks" style="display: inline; background-image: none;" alt="Import data from Azure blob storage into Databricks" src="https://zhenyuan.azurewebsites.net/posts/files/b67901b8-d2b2-48b3-8fad-c06881785917.png" border="0"&gt;&lt;/a&gt;&lt;p&gt;Data pipelines are essential for modern data solutions, and Azure Data Factory (ADF) provides a robust platform for building them. In this blog, we’ll walk through the process of setting up a pipeline in Azure Data Factory to import data from Azure Blob Storage into Databricks for processing.&lt;p&gt;&lt;br&gt;&lt;h2&gt;Step 1: Prerequisites&lt;/h2&gt;&lt;p&gt;Before setting up the pipeline, ensure the following prerequisites are met:&lt;ol&gt;&lt;li&gt;&lt;strong&gt;Azure Blob Storage&lt;/strong&gt;: Your source data should be stored in an Azure Blob Storage container.
&lt;li&gt;&lt;strong&gt;Azure Databricks Workspace&lt;/strong&gt;: A Databricks workspace and cluster should be set up for data processing.
&lt;li&gt;&lt;strong&gt;Azure Data Factory Instance&lt;/strong&gt;: Have an ADF instance provisioned in your Azure subscription.
&lt;li&gt;&lt;strong&gt;Linked Services Configuration&lt;/strong&gt;:
&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Azure Blob Storage Linked Service&lt;/strong&gt;: This enables ADF to connect to your data source.
&lt;li&gt;&lt;strong&gt;Azure Databricks Linked Service&lt;/strong&gt;: This enables ADF to connect to the target Databricks Delta Lake.&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;Both linked services are critical for establishing connections and configuring data pipelines between Blob Storage and Databricks.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/c0285ead-952e-4344-b9d2-66dfe63e10d8.png"&gt;&lt;img width="475" height="73" title="1" style="display: inline; background-image: none;" alt="1" src="https://zhenyuan.azurewebsites.net/posts/files/44ab22a5-e2b5-463c-9775-5567537cb716.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;ol start="5"&gt;&lt;li&gt;&lt;strong&gt;Access Permissions&lt;/strong&gt;:
&lt;/li&gt;&lt;ul&gt;&lt;li&gt;ADF needs Contributor access to Blob Storage and Databricks.
&lt;li&gt;Ensure you have access to generate a Databricks personal access token.&lt;/li&gt;&lt;/ul&gt;&lt;/ol&gt;&lt;p&gt;You will also need to configure the Blob Storage access token in Databricks. This ensures the underlying Spark cluster can connect to the source data seamlessly. Without proper configuration, you may encounter errors like the one shown below. &lt;/p&gt;&lt;p&gt;&lt;em&gt;&lt;font color="#ff0000"&gt;ErrorCode=AzureDatabricksCommandError,Hit an error when running the command in Azure Databricks. Error details: &amp;lt;span class='ansi-red-fg'&amp;gt;Py4JJavaError&amp;lt;/span&amp;gt;: An error occurred while calling o421.load.&lt;br&gt;
: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container salesdata in account cxitxstorage.blob.core.windows.net not found, and we can't create it using anoynomous credentials, and no credentials found for them in the configuration.&lt;br&gt;
Caused by: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container salesdata in account cxitxstorage.blob.core.windows.net not found, and we can't create it using anoynomous credentials, and no credentials found for them in the configuration.&lt;br&gt;
Caused by: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: Container salesdata in account cxitxstorage.blob.core.windows.net not found, and we can't create it using anoynomous credentials, and no credentials found for them in the configuration..&lt;/font&gt;&lt;/em&gt;&lt;/p&gt;&lt;p&gt;&lt;em&gt;&lt;font color="#ff0000"&gt;&lt;br&gt;&lt;/font&gt;&lt;/em&gt;&lt;/p&gt;&lt;p&gt;&lt;em&gt;&lt;font color="#ff0000"&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/4b9686d0-3034-4455-9f2a-56440e118e4f.png"&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/846f4a5e-f9d8-4bbe-a955-9dc55a0415cc.png"&gt;&lt;img width="435" height="554" title="2" style="display: inline; background-image: none;" alt="2" src="https://zhenyuan.azurewebsites.net/posts/files/621ccecf-37d3-4587-abd9-0462e91571fa.png" border="0"&gt;&lt;/a&gt;&lt;/font&gt;&lt;/em&gt;&lt;/p&gt;&lt;p&gt;In the Cluster Configuration you will need the following:&lt;p&gt;spark.hadoop.fs.azure.account.key.&amp;lt;account_name&amp;gt;.blob.core.windows.net {{secrets/&amp;lt;secret-scope-name&amp;gt;/&amp;lt;secret-name&amp;gt;}}&lt;br&gt;&lt;br&gt;&lt;p&gt;You can use Databricks CLI to create scope and secret with the below commands&lt;ul&gt;&lt;li&gt;databricks secrets create-scope &amp;lt;scope name&amp;gt;&lt;/li&gt;&lt;li&gt;databricks secrets put-secret --json '{&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; "scope": "&amp;lt;scope name&amp;gt;",&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; "key": "&amp;lt;secret-name&amp;gt;",&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; "string_value": "&amp;lt;storage account key value&amp;gt;",&lt;br&gt;
}&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;br&gt;&lt;h2&gt;&lt;font style="font-weight: normal;"&gt;Step 2: Create an ADF Pipeline&lt;/font&gt;&lt;/h2&gt;&lt;ol&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Add a Copy Data Activity&lt;/strong&gt;:&lt;ul&gt;&lt;li&gt;Add the &lt;strong&gt;Copy Data&lt;/strong&gt; activity into the pipeline.
&lt;li&gt;Set the &lt;strong&gt;Source&lt;/strong&gt; to use the Blob Storage dataset and the &lt;strong&gt;Sink&lt;/strong&gt; to use the Databricks Delta Lake dataset.&lt;/li&gt;&lt;/ul&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Add Data Transformation&lt;/strong&gt; (Optional):&lt;ul&gt;&lt;li&gt;Use a &lt;strong&gt;Databricks Notebook&lt;/strong&gt; activity to run transformation scripts.
&lt;li&gt;Link the notebook to your Databricks cluster and specify the notebook path.&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ol&gt;&lt;h2&gt;&lt;font style="font-weight: normal;"&gt;Step 3: Test and Schedule the Pipeline&lt;/font&gt;&lt;/h2&gt;&lt;ol&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Test the Pipeline&lt;/strong&gt;:&lt;ul&gt;&lt;li&gt;Use the &lt;strong&gt;Debug&lt;/strong&gt; feature in ADF to run the pipeline and verify its functionality.&lt;/li&gt;&lt;/ul&gt;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Schedule the Pipeline&lt;/strong&gt;:&lt;ul&gt;&lt;li&gt;Add a time-based or event-based trigger to automate pipeline runs.&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/d6cb7411-d075-4b8a-b363-7a55e4900f9c.png"&gt;&lt;img title="3" style="display: inline; background-image: none;" alt="3" src="https://zhenyuan.azurewebsites.net/posts/files/f932e259-2286-4505-b041-bd58bf6eb836.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;</description><pubDate>Fri, 22 Nov 2024 04:18:00 Z</pubDate><a10:updated>2024-11-22T04:18:01Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/unleashing-gpt-4-omni:-harnessing-data-insights-with-ai</guid><link>https://zhenyuan.azurewebsites.net/post/unleashing-gpt-4-omni:-harnessing-data-insights-with-ai</link><a10:author><a10:name /></a10:author><title>Unleashing GPT-4 Omni: Harnessing Data Insights with AI</title><description>&lt;p&gt;&lt;img alt="Harnessing Data Insights with AI" src="https://designerapp.officeapps.live.com/designerapp/document.ashx?path=/14e4e184-24f9-4000-90eb-4cd1756cca1a/DallEGeneratedImages/dalle-8f0cd79c-9a4c-49d5-a18a-bb27f068995a0251682919065680839400.jpg&amp;amp;dcHint=JapanEast&amp;amp;fileToken=292c1ec5-31be-4e2e-a715-5ec74937d684"&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;In the fast-evolving landscape of artificial intelligence, GPT-4 Omni stands at the forefront, promising not just advanced language processing capabilities, but also the potential to revolutionize how businesses derive insights from their data. Imagine a scenario where your finance team or C-level managers can seamlessly interact with your organization's data using natural language, thanks to the integration of GPT-4 Omni into your systems.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The Power of GPT-4 Omni in Data Analysis&lt;/strong&gt;&lt;br&gt;
GPT-4 Omni, developed by OpenAI, represents a significant leap forward in AI technology. Unlike its predecessors, GPT-4 Omni is designed to handle a broader range of tasks, including complex data analysis and generation of insights. This capability makes it an ideal candidate for businesses looking to democratize data access and empower non-technical users to explore and understand data in real-time.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Addressing Ad-Hoc Requests with Azure OpenAI Chatbot&lt;/strong&gt;&lt;br&gt;
Imagine a typical scenario: your finance team needs immediate insights into recent sales trends, or a C-level manager requires a quick analysis of profitability drivers. With an Azure OpenAI chatbot powered by GPT-4 Omni, these ad-hoc requests can be addressed swiftly and effectively. The chatbot can interact with users in natural language, understanding nuanced queries and providing meaningful responses based on the data at hand.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Demo Application: Bringing Data Insights to Life&lt;/strong&gt;&lt;br&gt;
Recently, I developed a demo application to showcase the capabilities of GPT-4 Omni in the realm of data analytics. In this demo, I uploaded a CSV file containing a sample sales dataset, complete with sales dates, products, categories, and revenue figures. The goal was to demonstrate how GPT-4 Omni can transform raw data into actionable insights through simple conversational queries.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/cac26b97-0fa1-4e07-a8cb-3ce654d0b5ad.png"&gt;&lt;img width="554" height="473" title="Screenshot 2024-06-26 160337" style="display: inline; background-image: none;" alt="Screenshot 2024-06-26 160337" src="https://zhenyuan.azurewebsites.net/posts/files/c296b8b8-d230-43a5-b5cb-b58d45c281e1.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;

&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;How It Works: From Data Upload to Insights&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;You can watch the video here &lt;/p&gt;&lt;p&gt;&lt;iframe width="560" height="315" title="YouTube video player" src="https://www.youtube.com/embed/PZaYtVmcGdE?si=JfqiD_t9DUCQAmZD" frameborder="0" allowfullscreen="" referrerpolicy="strict-origin-when-cross-origin" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"&gt;&lt;/iframe&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Data Upload and Integration&lt;/strong&gt;: The CSV file was uploaded into the demo application, which then processed and integrated the data into a format accessible to GPT-4 Omni.&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conversational Queries:&lt;/strong&gt; Users interacted with the chatbot by asking questions such as:&lt;/li&gt;&lt;ul&gt;&lt;li&gt;"What are the top-selling products in the sales data?"&lt;/li&gt;&lt;li&gt;"Is there any correlation between unit price and quantity sold?"&lt;/li&gt;&lt;li&gt;"Are there any seasonal trends in the sales data?"&lt;/li&gt;&lt;/ul&gt;&lt;/ul&gt;&lt;p&gt;&lt;br&gt;&lt;strong&gt;
Natural Language Processing:&lt;/strong&gt; GPT-4 Omni processed these queries, utilizing its advanced natural language understanding capabilities to interpret the intent behind each question.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Insight Generation&lt;/strong&gt;: Based on the data provided, GPT-4 Omni generated insightful responses, presenting trends, correlations, and summaries in a clear and understandable manner.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The Role of Assistants API&lt;/strong&gt;&lt;br&gt;
The Assistants API plays a pivotal role in enhancing functionality and integration capabilities. It empowers developers to create AI assistants within their applications, enabling these assistants to intelligently respond to user queries using a variety of models, tools, and files. Currently, the Assistants API supports three key functionalities: Code Interpretation, File Search, and Function Calling. For more detailed information, refer to &lt;a href="https://learn.microsoft.com/en-us/azure/ai-services/openai/assistants-quickstart?tabs=command-line%2Ctypescript&amp;amp;pivots=programming-language-studio"&gt;Quickstart - Getting started with Azure OpenAI Assistants (Preview) - Azure OpenAI | Microsoft Learn&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;br&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;br&gt;
As AI continues to advance, tools like GPT-4 Omni and the Assistants API are reshaping the business landscape, particularly in the realm of data analytics. The ability to leverage AI-driven insights from your own data, through intuitive and conversational interfaces, represents a significant competitive advantage. Whether it's optimizing operations, identifying new market opportunities, or improving financial forecasting, GPT-4 Omni and the Assistants API open doors to a more data-driven and agile business environment.&lt;/p&gt;
&lt;p&gt;In conclusion, integrating GPT-4 Omni and leveraging the Assistants API into your data strategy not only enhances operational efficiency but also fosters a culture of data-driven decision-making across your organization. Embrace the future of AI-powered data insights and unlock new possibilities for growth and innovation.&lt;/p&gt;</description><pubDate>Wed, 26 Jun 2024 10:47:01 Z</pubDate><a10:updated>2024-06-26T10:47:01Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/build-a-predictive-model-with-azure-ml</guid><link>https://zhenyuan.azurewebsites.net/post/build-a-predictive-model-with-azure-ml</link><a10:author><a10:name /></a10:author><title>Build a predictive model with Azure ML</title><description>&lt;p&gt;In today's data-driven world, businesses are constantly seeking ways to leverage their data for insights that can drive better decision-making and outcomes. Predictive modelling has emerged as a powerful tool for extracting actionable insights from data, enabling organizations to anticipate trends, forecast outcomes, and make informed decisions. Azure Machine Learning (Azure ML), a cloud-based platform, offers a suite of tools and services designed to simplify the process of building, training, and deploying predictive models. In this blog post, I’ll explore how to harness the capabilities of Azure ML to build a predictive model, focusing on Automated ML, Designer, feature selection, and propensity modelling with two-class classification regression.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Automated ML&lt;/strong&gt;: Automated ML is a powerful feature of Azure ML that automates the process of building machine learning models. With Automated ML, you can quickly experiment with different algorithms, hyperparameters, and feature transformations to find the best-performing model for your dataset. By leveraging Automated ML, data scientists can save time and resources while still achieving high-quality results. In our predictive modelling journey, we'll start by utilizing Automated ML to explore various model configurations and identify the most promising candidates for further optimization.&lt;p&gt;&lt;img alt="Automated Machine learning" src="https://learn.microsoft.com/en-us/azure/machine-learning/media/concept-automated-ml/automl-concept-diagram2.png?view=azureml-api-2"&gt;&lt;p&gt;By default, the models are ordered by metric score as they complete. For this tutorial, the model that scores the highest based on the chosen AUC_weighted metric is at the top of the list.&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/ca858bf8-f04f-4df8-8d76-be9635120f5e.png"&gt;&lt;img width="717" height="375" title="111" style="display: inline; background-image: none;" alt="111" src="https://zhenyuan.azurewebsites.net/posts/files/5e3037af-f640-48d4-b949-ead3478acaf5.png" border="0"&gt;&lt;/a&gt;&lt;p&gt;navigates through the &lt;strong&gt;Details&lt;/strong&gt; and the &lt;strong&gt;Metrics&lt;/strong&gt; tabs to view the selected model's properties, metrics, and performance charts.&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/01110ec0-3773-4d31-b018-00d307749507.png"&gt;&lt;img width="730" height="379" title="222" style="display: inline; background-image: none;" alt="222" src="https://zhenyuan.azurewebsites.net/posts/files/6a6e5461-8b0b-4f45-ae4f-2c441a8faf87.png" border="0"&gt;&lt;/a&gt;&lt;p&gt;&lt;br&gt;&lt;p&gt;&lt;strong&gt;Feature Selection:&lt;/strong&gt; Feature selection plays a crucial role in building predictive models by identifying the most relevant variables that contribute to the model's performance. Azure ML offers several feature selection techniques, ranging from univariate methods to more advanced algorithms. I'll employ these techniques to identify the most informative features in our dataset, reducing dimensionality and improving the interpretability of our model.&lt;p&gt;The screenshots below display the top four features ranked by their importance, as automatically determined by Automated ML. In this example, the decision to purchase a bike is influenced significantly by factors such as car ownership, age, marital status, and commute distance. These features emerge as key determinants in predicting the outcome, providing valuable insights into the underlying patterns driving consumer behaviour.&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/c047ff27-3825-4e0d-a2df-e23aa1ed889e.png"&gt;&lt;img width="725" height="199" title="333" style="display: inline; background-image: none;" alt="333" src="https://zhenyuan.azurewebsites.net/posts/files/f106be2c-2b23-4a79-86de-59d97c7c58b1.png" border="0"&gt;&lt;/a&gt;&lt;p&gt;&lt;br&gt;&lt;p&gt;&lt;strong&gt;Designer&lt;/strong&gt;: Azure ML Designer is a drag-and-drop interface that allows users to visually create, edit, and execute machine learning pipelines. With Designer, even users without extensive programming experience can easily build sophisticated machine learning workflows. We'll leverage Designer to construct our predictive modelling pipeline, incorporating data pre-processing steps, feature engineering techniques, and model training algorithms. By using Designer, we can streamline the development process and gain valuable insights into our data.&lt;p&gt;&lt;br&gt;&lt;p&gt;Propensity Modelling with Two-Class Classification Regression: Propensity modelling is a specialized form of predictive modelling that aims to predict the likelihood of a binary outcome, such as whether a customer will purchase a product or churn from a service. In our case, I’ll focus on building a propensity model using two-class classification regression techniques with &lt;strong&gt;designer&lt;/strong&gt;. By training our model on historical data with known outcomes, we can predict the propensity of future observations to belong to a particular class. This information can then be used to target interventions or marketing campaigns effectively.&lt;p&gt;The diagram below illustrates the pipeline designed for training the propensity model using two-class logistic regression. This pipeline encapsulates the sequence of steps involved in preparing the data, selecting features, and training the model to predict binary outcomes. With each component carefully orchestrated, the pipeline ensures a systematic and effective approach to building the propensity model, empowering organizations to make informed decisions based on predictive insights.&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/84cd0d54-051c-40fa-9546-5a3cc1763654.png"&gt;&lt;img width="730" height="525" title="444" style="display: inline; background-image: none;" alt="444" src="https://zhenyuan.azurewebsites.net/posts/files/58a4ec8d-3bb8-4c31-a72d-ab6e0d8b1f17.png" border="0"&gt;&lt;/a&gt;&lt;p&gt;&lt;br&gt;&lt;p&gt;The screenshot below presents the evaluation results, highlighting that the dataset is imbalanced, which corroborates with the findings detected by Automated ML. This imbalance in the dataset indicates a discrepancy in the distribution of classes, which could potentially impact the model's performance. Understanding and addressing this imbalance is crucial for ensuring the model's accuracy and reliability in real-world applications.&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/82941a8e-999b-4a70-8ed7-0b74303ef6ee.png"&gt;&lt;img width="730" height="455" title="555" style="display: inline; background-image: none;" alt="555" src="https://zhenyuan.azurewebsites.net/posts/files/4f5c5619-5c53-4e13-a4a3-08ad334ccfa6.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;Below screenshot shows the data guardrails are run by Automated ML when automatic featurization is enabled. This is a sequence of checks over the input data to ensure high quality data is being used to train model. &lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/3e1d9c04-d63f-4785-bdf7-6563deef4b18.png"&gt;&lt;img width="730" height="255" title="666" style="display: inline; background-image: none;" alt="666" src="https://zhenyuan.azurewebsites.net/posts/files/17d0146d-d456-4a4a-b784-266f246456f0.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;: In this blog post, we've explored how to build a predictive model with Azure ML, leveraging Automated ML, Designer, feature selection, and propensity modelling techniques. By harnessing the power of Azure ML, organizations can unlock valuable insights from their data and make data-driven decisions with confidence. Whether you're a seasoned data scientist or a novice analyst, Azure ML provides the tools and capabilities you need to succeed in the era of predictive analytics. So why wait? Start building your predictive models with Azure ML today and unlock the full potential of your data.&lt;/p&gt;</description><pubDate>Tue, 09 Apr 2024 00:41:44 Z</pubDate><a10:updated>2024-04-09T00:41:45Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/build-custom-bot-with-microsoft-bot-framework</guid><link>https://zhenyuan.azurewebsites.net/post/build-custom-bot-with-microsoft-bot-framework</link><a10:author><a10:name /></a10:author><title>Learn how to build a AI conversational bot with Microsoft bot framework</title><description>&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;img alt="Diagram explaining the components of a conversation AI experience" src="https://bot-framework.azureedge.net/static/334210-be29347d1e/intercom-webui/v1.6.2/assets/landing-page/images/BotFrameworkDiagram.png"&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;Recently I’m building a bot for prototyping with &lt;a href="https://dev.botframework.com/"&gt;Microsoft bot framework&lt;/a&gt; by integrating with new Azure cognitive services (&lt;a href="https://learn.microsoft.com/en-us/azure/cognitive-services/language-service/"&gt;cognitive service for language&lt;/a&gt;&amp;nbsp; &amp;amp; &lt;a href="https://azure.microsoft.com/en-us/products/cognitive-services/question-answering/"&gt;question answering&lt;/a&gt;). Frankly speaking, Microsoft bot framework is really easy to use and does not require steep learning curve.&amp;nbsp;&amp;nbsp; Today, I would like to share some of my experiences of building a bot, so you can build yours more effectively. &lt;/p&gt;&lt;p&gt;In the following article, I’m going to explain the tools I used and how to you can use them during the building life cycle as shown below.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;img alt="Design timeline of a bot" src="https://bot-framework.azureedge.net/static/334210-be29347d1e/intercom-webui/v1.6.2/assets/landing-page/images/BotLifeCycle.png"&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Design&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Build&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Test&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Publish&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Connect&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Evaluate &lt;/strong&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;br&gt;&lt;/strong&gt;&lt;/p&gt;&lt;h2&gt;Design Phase&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;/h2&gt;&lt;h3&gt;Microsoft Whiteboard&lt;/h3&gt;&lt;p&gt;Brainstorming the goal of the bot by asking below questions &lt;/p&gt;&lt;ul&gt;&lt;li&gt;why you need a bot?&lt;/li&gt;&lt;li&gt;what problem you are trying to resolve? &lt;/li&gt;&lt;li&gt;how to measure the success of your bot?&lt;/li&gt;&lt;li&gt;…etc.&lt;/li&gt;&lt;/ul&gt;&lt;h3&gt;Microsoft OneNote&lt;/h3&gt;&lt;p&gt;Design a conversational flow for your chatbot and try to analyze an example chatbot flowchart.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/7f14d790-5478-4d98-b2ba-2b6b5ee3d778.png"&gt;&lt;img width="332" height="582" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/f367b462-6bde-4c34-9ee7-761ec4b87dad.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h2&gt;&lt;br&gt;&lt;/h2&gt;&lt;h2&gt;Build Phase&lt;/h2&gt;&lt;h3&gt;Microsoft Visual Studio 2022&lt;/h3&gt;&lt;p&gt;Microsoft bot framework supports both Node.js and c#. In my case, I’m using c#.&amp;nbsp; I would suggest to start getting familiar with the &lt;a href="https://learn.microsoft.com/en-us/azure/bot-service/bot-service-quickstart-create-bot?view=azure-bot-service-4.0&amp;amp;tabs=csharp%2Cvs"&gt;framework SDK&lt;/a&gt;. It is well documented with code examples.&amp;nbsp; In my case, I started project with echo bot, which is a very simple template. It helps you understand the events lifecycle in the bot, before starting to build the complicated business logic. &lt;/p&gt;&lt;p&gt;The Echobot template is a .net core project.&amp;nbsp; If you are familiar with c# code.&amp;nbsp; you will find the bot endpoint is an API controller.&amp;nbsp; By default,&amp;nbsp; the .net core application is running on IIS Express in which anonymous authentication is enabled.&amp;nbsp; That’s why when you are not providing the username and password in bot framework emulator, it will still work. &lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/140b3a99-ab14-499f-bd0e-28291cdade3b.png"&gt;&lt;img width="551" height="372" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/b848fb4d-75d5-4a1d-8425-bf8e96e85223.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;h3&gt;MS Teams dev Tool&lt;/h3&gt;&lt;p&gt;Microsoft provides a great tool for designing and building dialog cards. you can drag and drop the element into the canvas which will automatically generate the json with styling data.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/52c3818a-7707-439a-b06f-23e79d7ba047.png"&gt;&lt;img width="570" height="368" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/04f8393a-7b72-459e-9991-89a97aea7133.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;h3&gt;Azure Cognitive Service for Language&lt;/h3&gt;&lt;p&gt;Azure cognitive service for language is a managed service to add high-quality natural language capabilities, from sentiment analysis and entity extraction to automated question answering.&amp;nbsp; &lt;/p&gt;&lt;p&gt;With Azure bot framework SDK, it makes easier to call the Cognitive Service. Here is an example in c#.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/a291a864-2443-4fc3-bfc3-d69a1374133a.png"&gt;&lt;img width="650" height="149" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/c814f69c-1d84-4fac-a813-439da4af1ee1.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h3&gt;Azure Cognitive Service for Question Answering&lt;/h3&gt;&lt;p&gt;Azure Question answering provides cloud-based Natural Language Processing (NLP) that allows you to create a natural conversational layer over your data.&amp;nbsp; It is used to find the most appropriate answer for any input from your custom knowledge base of information. Here is an example in c# with Azure bot framework SDK.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/9c996ce6-8d08-4f0f-b98e-756059f0fa71.png"&gt;&lt;img width="648" height="122" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/f2c83b88-24b7-4d7c-9aac-c41ec5bdad33.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h2&gt;&lt;br&gt;&lt;/h2&gt;&lt;h2&gt;Test Phase&lt;/h2&gt;&lt;h3&gt;Bot Framework Emulator v4&lt;/h3&gt;&lt;p&gt;Microsoft provides bot Framework Emulator so that you can test or debug your bot locally. For testing your bot in the Emulator, you only need to configure the endpoint.&amp;nbsp; In my case, it’s the default &lt;a href="http://localhost:3897/api/messages"&gt;http://localhost:3897/api/messages&lt;/a&gt;. you can leave Microsoft API ID and Microsoft App password empty, if they are empty in you appsettings.json.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/21ef5497-56f8-4470-aa7b-fc7cc21e6cad.png"&gt;&lt;img width="593" height="393" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/09d446a3-2c78-477e-b2b5-ee89ef6ae93a.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;Once you completed, then click “save and connect”, you are ready to debug.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/eacfbdc7-fcc5-45c5-a9b0-67310fc4dd9a.png"&gt;&lt;img width="621" height="329" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/116b1691-c1f9-4318-9a34-a71dd5c36c1a.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h2&gt;&lt;br&gt;&lt;/h2&gt;&lt;h2&gt;Publish Phase&lt;/h2&gt;&lt;p&gt;Github&lt;/p&gt;&lt;p&gt;There are many tools you can use for your CI/CD process. i.e. Azure devOps.&amp;nbsp; In my case, I’m using &lt;a href="https://github.com/"&gt;Github&lt;/a&gt;.&amp;nbsp; I’m hosting my bot in Azure app service, in which it could natively connect with Github by a few configuration in deployment center. &lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/a3ba1f22-2171-4642-9ea7-ce71a3e39cc7.png"&gt;&lt;img width="522" height="476" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/056ad385-2eb5-415c-b537-790f3871cfc6.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;In addition, it’s important to keep you credentials secure. DON’T put into appsettings.json, you should use github actions secrets instead.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/28df9eb6-7ea6-45f8-a300-35f2b624f90b.png"&gt;&lt;img width="638" height="339" title="image" style="display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/3ddee5aa-666a-4cab-b0d5-add79f101de7.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;On your local, you should leverage environment variable for appsettings.json.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/dcf50118-6b34-46e6-ab71-f119b365e332.png"&gt;&lt;img width="432" height="65" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/eb38bf8d-27ed-4aee-9127-84db20a69799.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;Now, you are ready to deploy your bot into Azure.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;h2&gt;Connect Phase&lt;/h2&gt;&lt;h3&gt;MS Teams&lt;/h3&gt;&lt;p&gt;Azure bot supports different channels, Web chat, Microsoft Teams, Alex, Email, Facebook, Slack etc. The good news is that Microsoft have already done the hard job for you, so that you don’t need to worry about the message formatting from different channels.&amp;nbsp; It will automatically convert into the conversational json required by your message endpoint. All you need to do in just register your channel.&amp;nbsp; In my case, I have registered for MS Teams.&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/5914871b-5483-40b4-8692-42e1beaaa766.png"&gt;&lt;img width="649" height="65" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/48bf3640-730a-40a0-92f6-0821cad1619c.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;After you registered the Teams channel in azure, you will need to create a Teams App package (manifest.zip) for the bot. It will need to be uploaded and installed in Teams. &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;h3&gt;Ngrok&lt;/h3&gt;&lt;p&gt;Another tool i’m using here is Ngrok for debugging remotely.&amp;nbsp; Ngrok secure tunnels allow you to instantly open access to remote systems without touching any of your network settings or opening any ports on your router.&amp;nbsp; You can find more details &lt;a href="https://learn.microsoft.com/en-us/azure/bot-service/bot-service-debug-channel-ngrok?view=azure-bot-service-4.0"&gt;here&lt;/a&gt; for configuring ngrok.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;h2&gt;Evaluate Phase&lt;/h2&gt;&lt;h3&gt;Azure Monitor Log Analytics &lt;/h3&gt;&lt;p&gt;Enabling Azure logs for analyzing bot behavior with Kusto queries like below samples. there are more samples you can find &lt;a href="https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-telemetry-analytics-queries?view=azure-bot-service-4.0#analyzing-bot-behavior"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;h4&gt;&lt;u&gt;Number of users per specific period&lt;/u&gt;&lt;/h4&gt;&lt;p&gt;&lt;img alt="Sample chart of number of users per period." src="https://learn.microsoft.com/en-us/azure/bot-service/v4sdk/media/userscount.png?view=azure-bot-service-4.0"&gt;&lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/fc11d8a2-d149-41ac-8b11-ad987cee6b3b.png"&gt;&lt;img width="519" height="240" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/26f7cdc9-294c-41c5-baa5-7c6213aa99d8.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h4&gt;&lt;u&gt;Activity per period&lt;/u&gt;&lt;/h4&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/d4e6de6f-a370-45c9-903b-2d73cc28176d.png"&gt;&lt;img width="500" height="271" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/06f57717-527e-4a75-8ae1-51b1b90db3aa.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h3&gt;Power BI&lt;/h3&gt;&lt;p&gt;Lastly, you can use Power BI&amp;nbsp; to build a dashboard.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;h2&gt;Summary &lt;/h2&gt;&lt;p&gt;Microsoft bot framework is a comprehensive framework for building enterprise-grade conversational AI experiences.&amp;nbsp; It makes easy to integrate with Azure Cognitive Services for creating a bot with the ability to speak, listen, understand, and learn.&amp;nbsp; It allows you to create an AI experience that can extend your brand and keep you in control of your own data.&amp;nbsp;&amp;nbsp;&amp;nbsp; Most importantly, Microsoft have already provided a full list of tools to smooth the process of your building experience.&amp;nbsp; &lt;/p&gt;&lt;p&gt;yay! here is my bot, his name is Eric.&amp;nbsp; Compose a bot today so as to boost your customers’ experience. &lt;/p&gt;&lt;p&gt;&lt;a href="https://zhenyuan.azurewebsites.net/posts/files/099d8011-9ce7-4491-9946-328dcef02571.png"&gt;&lt;img width="495" height="444" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://zhenyuan.azurewebsites.net/posts/files/431dd072-6d87-4f0e-8690-609b44f5d9dd.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;</description><pubDate>Mon, 09 Jan 2023 08:51:37 Z</pubDate><a10:updated>2023-01-09T08:51:37Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/automating-performance-audit-using-google-lighthouse-with-azure</guid><link>https://zhenyuan.azurewebsites.net/post/automating-performance-audit-using-google-lighthouse-with-azure</link><a10:author><a10:name /></a10:author><title>Automating performance audit using Google Lighthouse with Azure</title><description>&lt;p&gt;Running performance audits on a public-facing website is essential, in the past the audits was conducted manually. Recently, I have been asked to propose a solution for generating the Google Lighthouse report automatically.&amp;nbsp; &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;font color="#ff0000"&gt;&lt;u&gt;What is lighthouse?&lt;/u&gt;&lt;/font&gt;&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;Lighthouse is an open-source tool that analyzes web apps and web pages, collecting modern performance metrics and insights on developer best practices. you can find the repo &lt;a href="https://github.com/GoogleChrome/lighthouse"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;It mentioned in its document that you can run the report automatically with Node CLI.&amp;nbsp; Great start!&amp;nbsp; Yet,&amp;nbsp; I can run it on my machine, but how I share the reports with other people i.e. business as well as integrating with powerBI for reporting purpose?&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;After googling around, I didn’t find anything useful. so I decided to come up with my own solution.&amp;nbsp; &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;font color="#ff0000"&gt;&lt;u&gt;Proposed solution&lt;/u&gt;&lt;/font&gt;&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;Boom! here is the proposed solution. &lt;/p&gt;&lt;p&gt;Building the report on azure build agent, and publishing the report into blob storage. Simple right?!&amp;nbsp; With this approach, there is no dedicated node server required. In addition, storing report in blob can be simply shared with stakeholders and integrated with PowerBI.&amp;nbsp; &lt;/p&gt;&lt;p&gt;Brilliant, the completed architectural diagram as shown below. it’s a small implementation, but it still follows Well-Architected framework.&amp;nbsp; &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/7b2eedbd-bdef-476b-a1c1-ff4d498f32e5.png"&gt;&lt;img width="730" height="324" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/ad73c62b-ef9b-4991-b2f1-eb11341af1b9.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;p&gt;&lt;strong&gt;&lt;br&gt;&lt;/strong&gt;&lt;/p&gt;&lt;strong&gt;Operationally Excellence&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;To trigger generating the reports via Azure devOps allows me to setup a scheduled pipeline.&amp;nbsp; it provides insight about when pipeline is being triggered and sent notification if it fails.&amp;nbsp; with code as infrastructure mindset, all code are managed in Azure git and deployed via CI/CD pipeline.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Security&lt;/strong&gt; &lt;/p&gt;&lt;p&gt;Integrating with Azure AD for authentication, and using RBAC for segregating duties within the team for performing the jobs i.e. update pipeline, setup scheduling.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Reliability&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;Microsoft guarantee at least 99.9% availability for Azure devOps service and using self-hosted agent as failover plan for high availability. &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Performance Efficiency&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;A single blob supports up to 500 requests per second. Since there will not have massively requests for my project, so I’m not worry about the performance at all.&amp;nbsp; Yet, if you want to tuning the performance for your project, you can always use CDN (content delivery network) to distribute operations on the blob. or you can even use block storage account, which provides a higher request rate, or IOPS.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Cost Optimization&lt;/b&gt;&lt;/p&gt;&lt;p&gt;Comparing with VM solution, I believe this solution deliver at scale with the lowest price. Storage only costs AUD $0.31 per GB. &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;Hopefully you like this solution or share your thoughts if you have better options. All comments/suggests are welcomed.&lt;/p&gt;</description><pubDate>Thu, 21 Jul 2022 11:28:26 Z</pubDate><a10:updated>2022-07-21T11:28:26Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/managed-identities-in-azure</guid><link>https://zhenyuan.azurewebsites.net/post/managed-identities-in-azure</link><a10:author><a10:name /></a10:author><title>Managed Identities in Azure</title><description>&lt;p&gt;The challenge in the past is that every time you are developing a new webapp or bot which requires authentication you will go through all the steps i.e. creating service principle, grant permissions, set credentials, store credentials on resources, rotate credentials, and etc..&amp;nbsp;&amp;nbsp; Now there is a better solution: &lt;strong&gt;Managed identities&lt;/strong&gt; for Azure resources.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;One of the examples where you can adopt the managed identities when you want to build an application using web application that accesses Azure blob storage without having to manage any credentials. &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;font color="#ff0000"&gt;How to create&lt;/font&gt;&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;Managed identities is using service principle under the hook. Once you created the user assigned identity in Azure portal the same way as you creating other Azure resources,&amp;nbsp; you can now going to the target resources i..e blob storage and assign the permission i.e. contributor role to the user assigned identity you just created.&amp;nbsp;&amp;nbsp; Now, last step is to go to the Azure resource where you want to access the target blob storage, i.e. azure function, and in Identity blade, you can add the user assigned identity that you just have created.&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;Below is the example demonstrates authenticating the BlobClient from the Azure.Storage.Blobs client library using the DefaultAzureCredential with a user assigned managed identity configured.&lt;/p&gt;
&lt;script src="https://gist.github.com/zqbe/2f5111ec5f9484678a74e0bb8fa7f733.js"&gt;&lt;/script&gt;&lt;p&gt;DefaultAzureCredential discovery mechanism allows you to run the code with your signed in account when you are using the code locally and automatically switching to use the user assigned identity when the code is deployed in azure. Please be noted that your local account will require to have the same permission as the user assigned identity in Azure.&lt;/p&gt;&lt;p&gt;Managed identities for Azure resources provide Azure services with an automatically managed identity in Azure Active Directory. Using a managed identity, you can authenticate to any service that supports Azure AD authentication without managing credentials. You can find the list of available services &lt;a href="https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/managed-identities-status"&gt;here&lt;/a&gt;.&lt;/p&gt;</description><pubDate>Tue, 21 Jun 2022 07:43:22 Z</pubDate><a10:updated>2022-06-21T07:43:22Z</a10:updated></item><item><guid isPermaLink="true">https://zhenyuan.azurewebsites.net/post/sitecore-9.3-marketing-automation</guid><link>https://zhenyuan.azurewebsites.net/post/sitecore-9.3-marketing-automation</link><a10:author><a10:name /></a10:author><title>Sitecore 9.3 Marketing Automation–Custom API</title><description>&lt;p&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/19c4d0ae-dec6-41e2-a43f-adfb0e7a0aa7.png"&gt;&lt;img title="image" style="display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/2acb6323-17da-4d2d-bcaa-7dee52c9f9db.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;Sitecore marketing automation provides a way to create automated online campaigns in Sitecore. A marketing automation campaign is a plan that allows you to determine how to interact with contacts based on their interaction behavior with the website.&amp;nbsp; &lt;/p&gt;&lt;p align="justify"&gt;The marketing automation is powerful, yet there are some challenges as stated below when you want to leverage for supporting the business.&amp;nbsp;&amp;nbsp; This article is to share my experience how to extend Sitecore marketing automation to fit your business goals and overcome these challenges.&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h2 align="justify"&gt;Challenges&lt;/h2&gt;&lt;p align="justify"&gt;&lt;strong&gt;Only support MVC applications, unless you purchase JSS license&lt;/strong&gt; &lt;/p&gt;&lt;p align="justify"&gt;With the out-of-box Sitecore marketing automation, it supports MVC application natively. However, you may want to leverage marketing automation with your single page app (SPAs)i.e. ReactJS, Angular applications etc. via APIs.&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;strong&gt;Tracking specific user behaviours&lt;/strong&gt; &lt;/p&gt;&lt;p align="justify"&gt;In addition, the out-of-the-box supports events like page view events , what if you want to track specific user actions i.e. tracking whom make quote on your website.&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h2 align="justify"&gt;Business requirements &lt;/h2&gt;&lt;p align="justify"&gt;Before discussing the solution, let’s have a look what business wants to achieve. &lt;/p&gt;&lt;p align="justify"&gt;&lt;font color="#4f81bd"&gt;&lt;em&gt;One day morning, business came to me and ask if there is anyway we can get our customers more engaged than ever before? like sending them a reminder when they made a quote on our website.&lt;/em&gt;&lt;/font&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;p align="justify"&gt;&lt;em&gt;&lt;font color="#4f81bd"&gt;&lt;br&gt;&lt;/font&gt;&lt;/em&gt;&lt;/p&gt;&lt;h2 align="justify"&gt;Technical requirements&lt;/h2&gt;&lt;p align="justify"&gt;Engagement plan&lt;/p&gt;&lt;p align="justify"&gt;Based on the business requirements, the first thing on top of my head is Sitecore marketing automation engagement plan. Someone makes a quote can be a goal that a user achieved on the website. Once they triggered a goal, we can send them a reminder email.&amp;nbsp; Obviously, we need another goal when a user finalized the quote, as we don’t want to keep sending them reminders.&amp;nbsp; &lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h3 align="justify"&gt;Sitecore Services API&lt;/h3&gt;&lt;p align="justify"&gt;We have engagement plan, now how are we going to trigger a goal? our site is not MVC application, it’s a single page application.&amp;nbsp; Thus, tracking is not natively enabled. You can use MVC controller, however, why I decided to implement Sitecore services API is because that it has dependency injection enabled, and security enforced.&amp;nbsp;&amp;nbsp; &lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h3 align="justify"&gt;EXM email template and custom token&lt;/h3&gt;&lt;p align="justify"&gt;A company branded email template is required. What else? The ultimate goal of the reminder email is to encourage user to finalize the quote by purchasing the service.&amp;nbsp; Adding a hyperlink i.e. “click here to complete your purchase” seems a good idea for increasing the conversion rate.&amp;nbsp; Thus, we will need quote no. which will be a custom token. &lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;h2 align="justify"&gt;Solution &lt;/h2&gt;&lt;p align="justify"&gt;Cool, now we know what business wants to achieve and what’s required from technical point of view.&amp;nbsp; &lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h3 align="justify"&gt;1. Let’s start with creating &lt;font color="#ff0000" style="font-weight: normal;"&gt;goals&lt;/font&gt;&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;div align="justify"&gt;&lt;strong&gt;Make a quote&lt;/strong&gt;&amp;nbsp; - when user make a quote on the website.&lt;/div&gt;&lt;/li&gt;&lt;li&gt;&lt;div align="justify"&gt;&lt;strong&gt;Complete a quote&lt;/strong&gt; – when user completed the payment&lt;/div&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;font size="5"&gt;2. I also created custom &lt;font color="#ff0000"&gt;events&lt;/font&gt; &lt;/font&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;font size="4"&gt;This allows to store custom values i.e. quote Id which can be retrieved with custom token in EXM&lt;/font&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;div align="justify"&gt;&lt;strong&gt;&lt;font size="4"&gt;Make quote event &lt;/font&gt;&lt;/strong&gt;&lt;/div&gt;&lt;/li&gt;&lt;li&gt;&lt;div align="justify"&gt;&lt;font size="4"&gt;&lt;strong&gt;Complete quote event&lt;/strong&gt;&lt;/font&gt;&lt;/div&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h3 align="justify"&gt;3. Next, design a user journey&lt;/h3&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/0656be2d-b9e3-4572-a42e-a187d310ce4c.png"&gt;&lt;img width="730" height="1015" title="engagement plan" style="display: inline; background-image: none;" alt="engagement plan" src="http://zhenyuan.azurewebsites.net/posts/files/20e8b7ce-c55b-48a1-8c83-544f0c0984b2.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h3 align="justify"&gt;&lt;br&gt;&lt;/h3&gt;&lt;h3 align="justify"&gt;4. Creating Service APIs&lt;/h3&gt;&lt;p align="justify"&gt;There are a few things you will need to consider&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;div align="justify"&gt;Session&lt;/div&gt;&lt;/li&gt;&lt;li&gt;&lt;div align="justify"&gt;Authentication&lt;/div&gt;&lt;/li&gt;&lt;li&gt;&lt;div align="justify"&gt;Dependency Injection&lt;/div&gt;&lt;/li&gt;&lt;/ul&gt;&lt;h4 align="justify"&gt;&lt;br&gt;&lt;/h4&gt;&lt;h4 align="justify"&gt;&lt;font color="#4f81bd"&gt;Session&lt;/font&gt;&lt;/h4&gt;&lt;p align="justify"&gt;Services API by default doesn’t support session. most likely you will get Analytics.Tracking is null error.&amp;nbsp; Thus,&amp;nbsp; you will need to add session handler to your router.&lt;/p&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/52c993fd-5021-4064-b7da-d504fdf31eca.png"&gt;&lt;img width="501" height="180" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/b237bce8-026f-48fd-8a4e-f9523a5cc463.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h4 align="justify"&gt;&lt;font color="#4f81bd"&gt;Authentication&lt;/font&gt;&lt;/h4&gt;&lt;p align="justify"&gt;The good news is you can leverage Sitecore built-in Services API authentication for your custom controller.&amp;nbsp; you can find more details on how to authenticate your user &lt;a href="https://doc.sitecore.com/xp/en/developers/93/sitecore-experience-manager/the-restful-api-for-the-itemservice.html"&gt;here&lt;/a&gt;.&amp;nbsp; I would suggest to create specific user for your application. &lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h4 align="justify"&gt;&lt;font color="#4f81bd"&gt;Dependency Injection&lt;/font&gt;&lt;/h4&gt;&lt;p align="justify"&gt;Dependency injection allows you to loose coupling your code. Rather than instantiating objects directly in the code, often classes will declare their dependencies through constructor.&amp;nbsp;&amp;nbsp;&amp;nbsp; Sitecore services API allows you configure your injection in configuration. you can find more details &lt;a href="https://doc.sitecore.com/xp/en/developers/93/sitecore-experience-manager/dependency-injection.html"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;pre&gt;&amp;lt;configuration&amp;gt;
    &amp;lt;sitecore&amp;gt;
        &amp;lt;services&amp;gt;
            &amp;lt;register serviceType="IMyService, MyAbstraction.Assembly" implementationType="MyServiceImplementation, MyImplementation.Assembly" /&amp;gt;
            &amp;lt;register serviceType="MyServiceBase, MyAbstraction.Assembly" implementationType="MyServiceBaseImplementation, MyImplementation.Assembly" /&amp;gt;
        &amp;lt;/services&amp;gt;
    &amp;lt;/sitecore&amp;gt;
&amp;lt;/configuration&amp;gt;&lt;/pre&gt;&lt;p align="justify"&gt;&lt;font color="#4f81bd"&gt;&lt;br&gt;&lt;/font&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;font color="#4f81bd"&gt;Implement Triggering a goal&lt;/font&gt;&lt;/p&gt;&lt;p align="justify"&gt;Cool, by now if you are following the above steps you should have your API up and running with authentication enabled.&amp;nbsp; next is to create action for triggering the goal. &lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h5 align="justify"&gt;&lt;font color="#4f81bd"&gt;Identify contact&lt;/font&gt;&lt;/h5&gt;&lt;p align="justify"&gt;In my case, I need to identify the contact and update their information. IdentifyAs is the method I’m using. This basically tells sitecore that If it doesn’t exist then creating a new contact, otherwise makes it as known contact. &lt;/p&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/93f1bd95-9ada-451e-a50d-56572c20b66d.png"&gt;&lt;img width="565" height="36" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/7fbc1068-6672-4aee-8e03-57520863b626.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h5 align="justify"&gt;&lt;font color="#4f81bd"&gt;update contact&lt;/font&gt;&lt;/h5&gt;&lt;p align="justify"&gt;After contact is created, next is to update the contact details i.e. addressList, &lt;/p&gt;&lt;p align="justify"&gt;For updating contact, I’m using xConnect Client API.&amp;nbsp; Sitecore xConnect client API allows trusted clients to create, read, update, and search contacts and interactions over HTTPs.&amp;nbsp; you can find more details &lt;a href="https://doc.sitecore.com/xp/en/developers/93/sitecore-experience-platform/xconnect-client-api-overview.html"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;p align="justify"&gt;Don’t forget to extend the facets you want to update, as by default, sitecore doesn’t return most of the facets. Here is an example of extending facets.&lt;/p&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/212b35a9-b979-45a6-b37e-1adf3bf0886f.png"&gt;&lt;img width="567" height="220" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/11735a66-2ebb-4c31-b24e-40d27f354a4c.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h5 align="justify"&gt;&lt;font color="#4f81bd"&gt;triggering goal&lt;/font&gt;&lt;/h5&gt;&lt;p align="justify"&gt;It is a bit misleading about triggering a goal in Sitecore documentation. It states to register a goal for triggering it. Unfortunately, it won’t work with API, as it doesn’t have page context.&lt;/p&gt;&lt;pre&gt;var goalId = Guid.NewGuid(); // ID of goal
var goalDefinition = Sitecore.Analytics.Tracker.MarketingDefinitions.Goals[goalId];

Sitecore.Analytics.Tracker.Current.CurrentPage.RegisterGoal(goalDefinition);&lt;/pre&gt;&lt;p align="justify"&gt;Instead, I’ll still need to use xConnect Client API. Goal is specific type of an interaction. so you can add an interaction with a type of xConnect Goal&lt;/p&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/ca1ad352-8e3c-48a6-94c6-dc2031c40577.png"&gt;&lt;img width="730" height="74" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/71076573-9fc4-405f-8d34-890808d6561a.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;Once you added the goal, you are able to validate in the experience profile as shown below.&lt;/p&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/cfae5693-12a3-41ad-bf24-86f3590084b2.png"&gt;&lt;img width="730" height="265" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/97ce5fae-6c6b-4a9a-ba8c-a922f7d02259.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;p align="justify"&gt;Awesome, by now we have our goal triggered. we can start configure our engagement plan.&amp;nbsp; based on the business requirements,&amp;nbsp; I have created 3 reminders as shown below.&amp;nbsp; &lt;/p&gt;&lt;p align="justify"&gt;When users trigger “make a quote” goal, they will be engaged into the campaign, during the period, if they never complete their purchase, 3 reminders will be sent to them.&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/0ceac4ec-c5e0-4eaf-8bae-16cca37705cd.png"&gt;&lt;img width="663" height="795" title="image" style="display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/db75303b-dba1-4bd8-98a0-7d519b24d12d.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;Here is my testing results:&lt;/p&gt;&lt;p align="justify"&gt;I triggered two goals with different quote Id. Since I never completed the purchase, I have received 3 reminders for each of the quote.&lt;/p&gt;&lt;p align="justify"&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/88a751b8-33ad-4850-8eae-9596f4c5b40b.png"&gt;&lt;img width="611" height="168" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/d0cc9ebb-12e2-42e9-92ee-381b23a7de51.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p align="justify"&gt;&lt;br&gt;&lt;/p&gt;&lt;h2 align="justify"&gt;Summary&lt;/h2&gt;&lt;p align="justify"&gt;Hooray!&amp;nbsp; Now, the custom API extends Sitecore tracking capabilities to our SPAs and mobile apps.&amp;nbsp; As you can seen that Sky is the limit for using Sitecore Marketing Automation to achieve your business goals.&amp;nbsp;&amp;nbsp; I hope you enjoyed this article.&lt;/p&gt;</description><pubDate>Mon, 14 Mar 2022 11:08:33 Z</pubDate><a10:updated>2022-03-14T11:08:33Z</a10:updated></item><item><guid isPermaLink="false">https://zhenyuan.azurewebsites.net/post/issues-in-exercise–create-an-nva-and-virtual-machines</guid><link>https://zhenyuan.azurewebsites.net/post/issues-in-exercise%E2%80%93create-an-nva-and-virtual-machines</link><a10:author><a10:name /></a10:author><title>Issues in Exercise–Create an NVA and virtual machines</title><description>&lt;p&gt;If you are learning &lt;a href="https://docs.microsoft.com/en-us/learn/modules/control-network-traffic-flow-with-routes/5-exercise-create-nva-vm"&gt;AZ-303&lt;/a&gt;, and changes are you will encounter the same error when following the Exercise – Create an NVA and virtual machines (unit 5-7) by the time i’m writing this document (11/07/2021). &lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;I have reported this issue with the Exercise, but it might not be fixed anytime soon.&amp;nbsp; so just in case you need help with completing your Exercise. I tried to document the issue in Exercise, and the solution how you can resolve them.&lt;/p&gt;&lt;p&gt;&amp;nbsp;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/a836e0f3-4839-43b0-8582-4dd7b576de59.png"&gt;&lt;img width="632" height="143" title="image" style="display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/7e1aa897-def8-49d0-8b85-d2b1abdd65fd.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h2&gt;&lt;font color="#ff0000"&gt;&lt;u&gt;What’s the issue&lt;/u&gt;&lt;/font&gt;&lt;/h2&gt;&lt;p&gt;The requirement is to create vnent with 3 subnets &lt;/p&gt;&lt;ul&gt;&lt;li&gt;10.0.0.0/24&lt;/li&gt;&lt;li&gt;10.0.1.0/24&lt;/li&gt;&lt;li&gt;10.0.2.0/24&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/b6fca4b6-b829-4036-be00-f0891b466fc0.png"&gt;&lt;img width="519" height="330" title="image" style="display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/64a8beab-fe3d-4e93-a96b-651e1a6d763c.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&lt;p&gt;in Unit 5 of 7 , it provides the code as below for creating the 1st VM in subnet dmzsubnet.&amp;nbsp; The problem with the below command is it doesn’t specify the subnet address prefix, therefor by default, it will be 10.0.0.24.&amp;nbsp; You won’t yet get any error at this stage, as it’s the first subnet. Although it’s already not matching the design.&lt;/p&gt;&lt;p&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/bd6d975d-67e8-4791-ac43-d424728c1879.png"&gt;&lt;img width="524" height="156" title="image" style="display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/45e3d190-b2cf-4814-a59c-dfee4a66f54e.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;p&gt;You will then get an error when you are following the Exercise along the way in Unit 6 of 7 with below code.&amp;nbsp; you can’t create subnet with conflict address. &lt;/p&gt;&lt;p&gt;&lt;a href="http://zhenyuan.azurewebsites.net/posts/files/3decb971-f5fa-4486-afb2-4bc2e29d3d42.png"&gt;&lt;img width="540" height="187" title="image" style="display: inline; background-image: none;" alt="image" src="http://zhenyuan.azurewebsites.net/posts/files/077ebef9-b366-4bca-ba9c-1d05181263d9.png" border="0"&gt;&lt;/a&gt;&lt;/p&gt;&lt;h2&gt;&lt;font color="#ff0000"&gt;&lt;u&gt;What’s the Solution&lt;/u&gt;&lt;/font&gt;&lt;/h2&gt;&lt;p&gt;Adding&amp;nbsp; “--subnet-address-prefix” when you create VMs. There are more optional parameters for creating VM you can find &lt;a href="https://docs.microsoft.com/en-us/cli/azure/vm?view=azure-cli-latest#az_vm_create"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;&lt;p&gt;here is the example code you can use &lt;/p&gt;&lt;p&gt;az vm create \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --resource-group learn-4bd2e66c-7759-446c-9a49-071b27237a7f \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --name public \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --vnet-name vnet \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --subnet publicsubnet \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --image UbuntuLTS \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --admin-username azureuser \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --no-wait \&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --custom-data cloud-init.txt \&lt;br&gt;&amp;nbsp; &lt;font style="background-color: rgb(255, 255, 0);"&gt;&amp;nbsp;&amp;nbsp; --subnet-address-prefix 10.0.2.0/24 \&lt;/font&gt;&lt;br&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; --admin-password &amp;lt;changeme123&amp;gt;&lt;/p&gt;&lt;p&gt;&lt;u&gt;&lt;font color="#ff0000"&gt;&lt;br&gt;&lt;/font&gt;&lt;/u&gt;&lt;/p&gt;</description><pubDate>Sun, 11 Jul 2021 06:09:00 Z</pubDate><a10:updated>2021-07-11T06:09:00Z</a10:updated></item></channel></rss>