-
Notifications
You must be signed in to change notification settings - Fork 315
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to fix this issue for C# Winforms project #242
Comments
Here are some steps and tips to help you resolve this issue: 1. Tensor DimensionsThe error message indicates a mismatch in tensor dimensions. Specifically, the model expects a tensor of dimensions 2. Preprocess Image FunctionUpdate your private Tensor<float> PreprocessImage(string imagePath)
{
// Load the image
using var bitmap = new Bitmap(imagePath);
// Ensure the image is of size 854x480
const int targetWidth = 854;
const int targetHeight = 480;
// Convert to a Tensor<float> in NCHW format (batch, channels, height, width)
var tensor = new DenseTensor<float>(new[] { 1, 3, targetHeight, targetWidth }); // Batch, Channels, Height, Width
for (int y = 0; y < targetHeight; y++)
{
for (int x = 0; x < targetWidth; x++)
{
var pixel = bitmap.GetPixel(x, y);
// Normalize pixel values to [0, 1] and populate the tensor
tensor[0, 0, y, x] = pixel.R / 255f; // Red channel
tensor[0, 1, y, x] = pixel.G / 255f; // Green channel
tensor[0, 2, y, x] = pixel.B / 255f; // Blue channel
}
}
return tensor;
} 3. Prepare Vision InputsEnsure that the private List<NamedOnnxValue> PrepareVisionInputs(string imagePath)
{
// Preprocess the image
var imageTensor = PreprocessImage(imagePath);
// Prepare the image size tensor (Height and Width)
var imageSizeTensor = new DenseTensor<long>(new[] { 1, 2 });
imageSizeTensor[0, 0] = 480; // Height
imageSizeTensor[0, 1] = 854; // Width
// Debugging: Print the tensor dimensions and types
Debug.Print($"Image tensor dimensions: {string.Join(", ", imageTensor.Dimensions.ToArray())}");
Debug.Print($"Image size tensor dimensions: {string.Join(", ", imageSizeTensor.Dimensions.ToArray())}");
// Create inputs for the vision model
var inputs = new List<NamedOnnxValue>
{
NamedOnnxValue.CreateFromTensor("pixel_values", imageTensor),
NamedOnnxValue.CreateFromTensor("image_sizes", imageSizeTensor)
};
return inputs;
} 4. DebuggingAdd debugging statements to ensure the dimensions of your tensors are correct before passing them to the model: Debug.Print($"Image tensor dimensions: {string.Join(", ", imageTensor.Dimensions.ToArray())}");
Debug.Print($"Image size tensor dimensions: {string.Join(", ", imageSizeTensor.Dimensions.ToArray())}"); 5. Running the ApplicationEnsure that your private void BTNInfer_Click(object sender, EventArgs e)
{
using (OpenFileDialog openFileDialog = new OpenFileDialog())
{
openFileDialog.Filter = "Image Files|*.png;*.jpg;*.jpeg";
if (openFileDialog.ShowDialog() == DialogResult.OK)
{
string filePath = openFileDialog.FileName;
PBImage.Image = Image.FromFile(filePath);
DetectObjects(filePath);
}
}
} 6. Exception HandlingEnsure you have proper exception handling to catch and debug any runtime errors: private void DetectObjects(string imagePath)
{
try
{
// Preprocess the image
var imageTensor = PreprocessImage(imagePath);
// Prepare inputs for the vision model
var visionInputs = PrepareVisionInputs(imagePath);
// Run the vision model
var visualFeatures = RunVisionModel(visionInputs);
// Prepare inputs for the text model
var textInputs = PrepareTextInputs(visualFeatures);
// Run the text model
var response = RunTextModel(textInputs);
// Display the response
MessageBox.Show($"Model Response: {response}");
}
catch (Exception ex)
{
Debug.Print(ex.Message.ToString());
MessageBox.Show($"Error during inference: {ex.Message}");
}
} By following these steps, you should be able to resolve the tensor dimension mismatch and successfully run your application to describe images using the Phi-3 model. |
Hi, I always get the same run time errors: Image tensor dimensions: 1, 1, 3, 480, 854
Image size tensor dimensions: 1, 2
Exception thrown: 'Microsoft.ML.OnnxRuntime.OnnxRuntimeException' in Microsoft.ML.OnnxRuntime.dll
[ErrorCode:RuntimeException] Non-zero status code returned while running Add node. Name:'/img_processor/vision_model/embeddings/Add' Status Message: D:\a_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:560 onnxruntime::BroadcastIterator::Append axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 577 by 2075
=> I write one WinForms app project, I want to use Phi-3 model to describe the image, but my image size is different: 854 by 480 pixels.
Here is my form1.cs code: using System;
using System.Diagnostics;
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Microsoft.ML.OnnxRuntime;
using Microsoft.ML.OnnxRuntime.Tensors;
using Newtonsoft.Json.Linq;
namespace TestPhi3Form
{
public partial class Form1 : Form
{
private string visionModelPath =
@"D:\Phi3\Phi3-vision-128k-instruct-onnx-cpu\Phi3-vision-128k-instruct-onnx-cpu\phi-3-v-128k-instruct-vision.onnx";
private string textModelPath =
@"D:\Phi3\Phi3-vision-128k-instruct-onnx-cpu\Phi3-vision-128k-instruct-onnx-cpu\phi-3-v-128k-instruct-text.onnx";
private string tokenPath =
@"D:\Phi3\Phi3-vision-128k-instruct-onnx-cpu\Phi3-vision-128k-instruct-onnx-cpu\tokenizer.json";
}
=> I want to know how to run my program, so I can click on BTNInfer button to open any image file in my PC and let Phi-3 to describe what it can see in the image.
Please advise,
Thanks,
The text was updated successfully, but these errors were encountered: