Quantum Machine Learning with Python: Kernel Methods and Neural Networks

Introduction

Quantum Machine Learning (QML) represents a fascinating convergence of quantum computing and machine learning technologies. With quantum computing’s potential in mathematics and data processing with complex structure, QML could revolutionize areas like drug discovery, finance, and beyond. This blog delves into the innovative realms of quantum neural networks (QNNs) and quantum kernel techniques, showcasing their unique capabilities through practical Python examples. The blog will not detail the mathematical concepts. For more information do not hesitate to read my latest book Machine Learning Theory and Applications: Hands-on Use Cases with Python on Classical and Quantum Machines, Wiley, 2024.

Quantum kernel methods, introduce a quantum-enhanced way of processing data. By mapping classical data into quantum feature space, these methods utilize the superposition and entanglement properties of quantum mechanics to perform classifications or regression tasks. The use of quantum kernel estimator and quantum variational classifier examples illustrates the practical application of these concepts. QNNs, leveraging quantum states for computation, offer a novel approach to neural network architecture. The Qiskit framework facilitates the implementation of both quantum kernel methods and QNNs, enabling the exploration of quantum algorithms’ efficiency in learning and pattern recognition.

Incorporating Python code examples, this blog aims to provide comprehensive code examples of QML for readers to explore its promising applications, and the challenges it faces. Through these examples, readers can start practicing and gain an appreciation for the transformative potential of quantum computing in machine learning and the exciting possibilities that lie ahead.

First, install qiskit

We will use the open-source SDK Qiskit (https://qiskit.org) which allows working with quantum computers. Qiskit supports Python version 3.6 or later.

In our environment, we can install Qiskit with pip:

pip install qiskit

We can also install qiskit-machine-learning using pip:

pip install qiskit-machine-learning

Documentation can be found on GitHub: https://github.com/Qiskit/qiskit-machine-learning/.

To run our code, we can use either simulators or real hardware even if I strongly recommend the use of hardware or push the limits of simulators to improve research in this field. While studying the Qiskit documentation, you will encounter references to the Qiskit Runtime primitives, which serve as implementations of the Sampler and Estimator interfaces found in the qiskit.primitives module. These interfaces facilitate the seamless interchangeability of primitive implementations with minimal code modifications. The initial release of Qiskit Runtime comprises two essential primitives:

• Sampler: This primitive generates quasi-probabilities based on input circuits.
• Estimator: This primitive calculates expectation values derived from input circuits and observables.

For more comprehensive insights, detailed information is available in the following resource: https://qiskit.org/ecosystem/ibm-runtime/tutorials/how-to-getting-started-with-sampler.html.

Quantum Kernel Methods

Venturing into quantum approaches for supervised machine learning poses a novel research direction. Classical machine learning extensively utilizes kernel methods, among which the support vector machine (SVM) for classification stands out for its widespread application.

SVMs, known for their role in binary classification, have increasingly been applied to multiclass problems. The essence of binary SVM involves devising a hyperplane to linearly separate n-dimensional data points into two groups, aiming for an optimal margin that distinctively classifies the data into its respective categories. This hyperplane, effective in either the original feature space or a transformed higher-dimensional kernel space, is selected for its capacity to maximize the separation between classes, which involves an optimization problem to maximize the margin, defined as the distance from the nearest data point to the hyperplane on either side. This leads to the formulation of a maximum-margin classifier. The critical data points on the boundary are termed support vectors, and the margin represents a zone typically devoid of data points. An optimal hyperplane too proximate to the data points, indicating a slender margin, undermines the model’s predictive robustness and generalization capability.

To navigate multiclass SVM challenges, methods like the all-pair strategy, which conducts a binary classification for each pair of classes, have been introduced. Beyond straightforward linear classification, nonlinear classifications can be achieved through the kernel trick. This technique employs a kernel function to elevate inputs into a more expansive, higher-dimensional feature space, facilitating the separation of data that is not linearly separable in the input space. The kernel function essentially performs an inner product in a potentially vast Euclidian space, known as the feature space. The goal of nonlinear SVM is to achieve this separation by mapping data to a higher dimension using a suitable mapping. Selecting an appropriate feature map becomes crucial for data that cannot be addressed by linear methods alone. This is where quantum can jump into it. Quantum kernel methods, blending classical kernel strategies with quantum innovations, carve out new avenues in machine learning. Early quantum kernel approaches have focused on encoding data points into inner products or amplitudes in Hilbert space through quantum feature maps. The complexity of the quantum circuit implementing the feature map scales linearly or polylogarithmically with the dataset size.

Quantum Kernel with ZZFeatureMaps

In this first example, we will use the ZZFeatureMap with linear entanglement, we will repeat the data encoding step two times, and we will use feature reduction with principal component analysis. You can of course use other feature reduction, data rescaling or feature selection techniques to improve the accuracy of your models. We will use the breast cancer dataset that you can find here: https://github.com/xaviervasques/hephaistos/blob/main/data/datasets/breastcancer.csv

Let’s describe the steps of the Python script below. This Python script demonstrates an application of integrating quantum computing techniques with traditional machine learning to classify breast cancer data. It represents a hybrid approach, where quantum-enhanced features are used within a classical machine learning workflow. The goal is to predict breast cancer diagnosis (benign or malignant) based on a set of features extracted from the breast mass characteristics.

The way of doing quantum kernel machine learning is very similar to what we do classically as data scientists. We import the necessary libraries (Pandas, NumPy, scikit-learn) and Qiskit for quantum computing and kernel estimation, we load the data, preprocess the data and separate the data into features (X) and target labels (y). A specific step is the quantum feature mapping. The script sets up a quantum feature map using the ZZFeatureMap from Qiskit, configured with specified parameters for feature dimension, repetitions, and entanglement type. Quantum feature maps are critical for translating classical data into quantum states, enabling the application of quantum computing principles for data analysis. Then, the quantum kernel setup consists in configuring a quantum kernel with a fidelity-based approach. It serves as a new method to compute the similarity between data points in the feature space defined by quantum states and potentially capturing complex patterns. The last step comes back to a classic machine learning pipeline with data rescaling with standard scaler, dimension reduction using principal component analysis and the use of support vector classifier (SVC) which utilizes the quantum kernel for classification. We evaluate the model using 5-fold cross-validation.

Let’s code.

# Import necessary libraries for data manipulation, machine learning, and quantum computing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder

# Load the dataset using pandas, specifying the file location and delimiter
breastcancer = './breastcancer.csv'
df = pd.read_csv(breastcancer, delimiter=';')

# Remove the 'id' column as it is not useful for prediction, to simplify the dataset
df = df.drop(["id"], axis=1)

# Separate the dataset into features (X) and target label (y)
y = df['diagnosis']  # Target label: diagnosis
X = df.drop('diagnosis', axis=1)  # Features: all other columns

# Convert the diagnosis string labels into numeric values to be used by machine learning models
label_encoder = LabelEncoder()
y = label_encoder.fit_transform(y)

# Quantum computing sections start here
# Set parameters for the quantum feature map
feature_dimension = 2  # Number of features used in the quantum feature map
reps = 2  # Number of repetitions of the feature map circuit
entanglement = 'linear'  # Type of entanglement in the quantum circuit

# Import quantum feature mapping utilities from Qiskit
from qiskit.circuit.library import ZZFeatureMap
qfm = ZZFeatureMap(feature_dimension=feature_dimension, reps=reps, entanglement=entanglement)

# Set up a local simulator for quantum computation
from qiskit.primitives import Sampler
sampler = Sampler()

# Configure quantum kernel using ZZFeatureMap and a fidelity-based quantum kernel
from qiskit.algorithms.state_fidelities import ComputeUncompute
from qiskit_machine_learning.kernels import FidelityQuantumKernel
fidelity = ComputeUncompute(sampler=sampler)
quantum_zz = FidelityQuantumKernel(fidelity=fidelity, feature_map=qfm)

# Create a machine learning pipeline integrating standard scaler, PCA for dimensionality reduction,
# and a Support Vector Classifier using the quantum kernel
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.svm import SVC
pipeline = make_pipeline(StandardScaler(), PCA(n_components=2), SVC(kernel=quantum_zz.evaluate))

# Evaluate the model using cross-validation to assess its performance
from sklearn.model_selection import cross_val_score
cv = cross_val_score(pipeline, X, y, cv=5, n_jobs=1)  # n_jobs=1 specifies that the computation will use 1 CPU
mean_score = np.mean(cv)  # Calculate the mean of the cross-validation scores

# Print the mean cross-validation score to evaluate the model's performance
print(mean_score)

We will obtain a mean score validation score of 0.63.

This code is executed with the local simulator. To run on real hardware, replace the following lines:

# Set up a local simulator for quantum computation
from qiskit.primitives import Sampler
sampler = Sampler()

by

# Import necessary classes from qiskit_ibm_runtime for accessing IBM Quantum services
from qiskit_ibm_runtime import QiskitRuntimeService, Sampler

# Initialize the QiskitRuntimeService with your IBM Quantum credentials
# 'channel', 'token', and 'instance' are placeholders for your actual IBM Quantum account details
service = QiskitRuntimeService(channel='YOUR CHANNEL', token='YOUR TOKEN FROM IBM QUANTUM', instance='YOUR INSTANCE')

# Specify the backend you wish to use. This could be a simulator or an actual quantum computer available through IBM Quantum
# 'quantum_backend' should be replaced with the name of the quantum backend you wish to use
backend = service.backend('quantum_backend')

# Import the Options class to customize the execution of quantum programs
from qiskit_ibm_runtime import Options
options = Options()  # Create an instance of Options

# Set the resilience level. Level 1 typically implies some level of error mitigation or resilience against errors
options.resilience_level = 1

# Set the number of shots, which is the number of times the quantum circuit will be executed to gather statistics
# More shots can lead to more accurate results but take longer to execute
options.execution.shots = 1024

# Set the optimization level for compiling the quantum circuit
# Higher optimization levels attempt to reduce the circuit's complexity, which can improve execution but may take longer to compile
options.optimization_level = 3

# Initialize the Sampler, which is used to run quantum circuits and obtain samples from their measurement outcomes
# The Sampler is configured with the specified backend and options
sampler = Sampler(session=backend, options=options)

Quantum Kernel Training

This part will explore the method of Quantum Kernel Alignment (QKA) for the purpose of binary classification. QKA iteratively adjusts a quantum kernel that’s parameterized to fit a dataset, aiming for the largest possible margin in Support Vector Machines (SVM). For further details on QKA, reference is made to the preprint titled “Covariant quantum kernels for data with group structure.” The Python script below is a comprehensive example of integrating traditional machine learning techniques with quantum computing for the prediction accuracy in classifying breast cancer diagnosis. It employs a dataset of breast cancer characteristics to predict the diagnosis (benign or malignant).

The machine learning pipeline is similar to the one used in the quantum kernel with ZZFeatureMaps section. The difference is that we will constructs a custom quantum circuit, integrating a rotational layer with a ZZFeatureMap, to prepare the quantum state representations of the data. The quantum kernel estimation step utilizes Qiskit primitives and algorithms for optimizing the quantum kernel’s parameters using a quantum kernel trained (QKT) and an optimizer.

Let’s code.

# Import necessary libraries for data manipulation, machine learning, and quantum computing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder

# Load the dataset using pandas, specifying the file location and delimiter
breastcancer = './breastcancer.csv'
df = pd.read_csv(breastcancer, delimiter=';')

# Remove the 'id' column as it is not useful for prediction, to simplify the dataset
df = df.drop(["id"], axis=1)

# Reduce the dataframe size by sampling 1/3 of the data
df = df.sample(frac=1/3, random_state=1)  # random_state for reproducibility

# Separate the dataset into features (X) and target label (y)
y = df['diagnosis']  # Target label: diagnosis
X = df.drop('diagnosis', axis=1)  # Features: all other columns

# Convert the diagnosis string labels into numeric values to be used by machine learning models
label_encoder = LabelEncoder()
y = label_encoder.fit_transform(y)

# Quantum computing sections start here
# Set parameters for the quantum feature map
feature_dimension = 2  # Number of features used in the quantum feature map
reps = 2  # Number of repetitions of the feature map circuit
entanglement = 'linear'  # Type of entanglement in the quantum circuit

# Define a custom rotational layer for the quantum feature map
from qiskit import QuantumCircuit
from qiskit.circuit import ParameterVector
training_params = ParameterVector("θ", 1)
fm0 = QuantumCircuit(feature_dimension)
for qubit in range(feature_dimension):
    fm0.ry(training_params[0], qubit)

# Use ZZFeatureMap to represent input data
from qiskit.circuit.library import ZZFeatureMap
fm1 = ZZFeatureMap(feature_dimension=feature_dimension, reps=reps, entanglement=entanglement)

# Compose the custom rotational layer with the ZZFeatureMap to create the feature map
fm = fm0.compose(fm1)

# Initialize the Sampler, a Qiskit primitive for sampling from quantum circuits
from qiskit.primitives import Sampler
sampler = Sampler()

# Set up the ComputeUncompute fidelity object for quantum kernel estimation
from qiskit.algorithms.state_fidelities import ComputeUncompute
from qiskit_machine_learning.kernels import TrainableFidelityQuantumKernel
fidelity = ComputeUncompute(sampler=sampler)

# Instantiate the quantum kernel with the feature map and training parameters
quant_kernel = TrainableFidelityQuantumKernel(fidelity=fidelity, feature_map=fm, training_parameters=training_params)

# Callback class for tracking optimization progress
class QKTCallback:
    # Callback wrapper class
    def __init__(self):
        self._data = [[] for i in range(5)]

    def callback(self, x0, x1=None, x2=None, x3=None, x4=None):
        #Capture callback data for analysis
        for i, x in enumerate([x0, x1, x2, x3, x4]):
            self._data[i].append(x)

    def get_callback_data(self):
        #Get captured callback data
        return self._data

    def clear_callback_data(self):
        #Clear captured callback data
        self._data = [[] for i in range(5)]

# Setup and instantiate the optimizer for the quantum kernel
from qiskit.algorithms.optimizers import SPSA
cb_qkt = QKTCallback()
spsa_opt = SPSA(maxiter=10, callback=cb_qkt.callback, learning_rate=0.01, perturbation=0.05)

# Quantum Kernel Trainer (QKT) for optimizing the kernel parameters
from qiskit_machine_learning.kernels.algorithms import QuantumKernelTrainer
qkt = QuantumKernelTrainer(
    quantum_kernel=quant_kernel, loss="svc_loss", optimizer=spsa_opt, initial_point=[np.pi / 2]
)

# Reduce dimensionality of the data using PCA
from sklearn.decomposition import PCA
pca = PCA(n_components=2)
X_ = pca.fit_transform(X)

# Train the quantum kernel with the reduced dataset
qka_results = qkt.fit(X_, y)
optimized_kernel = qka_results.quantum_kernel

# Use the quantum-enhanced kernel in a Quantum Support Vector Classifier (QSVC)
from qiskit_machine_learning.algorithms import QSVC
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
qsvc = QSVC(quantum_kernel=optimized_kernel)
pipeline = make_pipeline(StandardScaler(), PCA(n_components=2), qsvc)

# Evaluate the performance of the model using cross-validation
from sklearn.model_selection import cross_val_score
cv = cross_val_score(pipeline, X, y, cv=5, n_jobs=1)
mean_score = np.mean(cv)

# Print the mean cross-validation score
print(mean_score)

We will obtain the following output: 0.6526315789473685

As you certainly observed, there is time differences in execution between QKT and using a quantum kernel with a predefined feature map like ZZFeatureMap even if we reduced the dataframe size by sampling 1/3 of the data and setting the maximum iteration for SPSA to 10. QKT involves not only the use of a quantum kernel but also the optimization of parameters within the quantum feature map or the kernel itself to improve model performance. This optimization process requires iterative adjustments to the parameters, where each iteration involves running quantum computations to evaluate the performance of the current parameter set. This iterative nature significantly increases computational time. When using a predefined quantum kernel like the ZZFeatureMap, the feature mapping is fixed, and there’s no iterative optimization of quantum parameters involved. The quantum computations are performed to evaluate the kernel between data points, but without the added overhead of adjusting and optimizing quantum circuit parameters. This approach is more straightforward and requires fewer quantum computations, making it faster. Each step of the optimization process in QKT requires evaluating the model’s performance with the current quantum kernel, which depends on the quantum feature map parameters at that step. This means multiple evaluations of the kernel matrix, each of which requires a substantial number of quantum computations.

Quantum Neural Networks

This Python script below incorporates quantum neural networks (QNNs) into a machine learning pipeline. In the script, we need to configure the quantum feature map and ansatz (a quantum circuit structure), construct a quantum circuit by appending the feature map and ansatz to a base quantum circuit (this setup is crucial for creating quantum neural networks that process input data quantum mechanically) and create a QNN using the quantum circuit designed for binary classification. Before coming back to the classic machine learning pipeline with data rescaling, data reduction and model evaluation, we employ a quantum classifier which integrates the QNN with a classical optimization algorithm (COBYLA) for training. A callback function is defined to visualize the optimization process, tracking the objective function value across iterations.

Let’s code.

# Importing essential libraries for handling data, machine learning, and integrating quantum computing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
import matplotlib.pyplot as plt  # For data visualization

# Load and prepare the dataset
breastcancer = './breastcancer.csv'
df = pd.read_csv(breastcancer, delimiter=';')  # Load dataset from CSV file
df = df.drop(["id"], axis=1)  # Remove the 'id' column as it's not necessary for analysis

# Splitting the data into features (X) and the target variable (y)
y = df['diagnosis']  # Target variable: diagnosis result
X = df.drop('diagnosis', axis=1)  # Feature matrix: all data except the diagnosis

# Encoding string labels in 'y' into numerical form for machine learning models
label_encoder = LabelEncoder()
y = label_encoder.fit_transform(y)  # Transform labels to numeric

# Quantum feature map and circuit configuration
feature_dimension = 2  # Dimensionality for the feature map (matches PCA reduction later)
reps = 2  # Number of repetitions of the ansatz circuit for depth
entanglement = 'linear'  # Type of qubit entanglement in the circuit

# Initialize an array to store evaluations of the objective function during optimization
objective_func_vals = []

# Define a callback function for visualization of the optimization process
def callback_graph(weights, obj_func_eval):
    """Updates and saves a plot of the objective function value after each iteration."""
    objective_func_vals.append(obj_func_eval)
    plt.title("Objective function value against iteration")
    plt.xlabel("Iteration")
    plt.ylabel("Objective function value")
    plt.plot(range(len(objective_func_vals)), objective_func_vals)
    plt.savefig('Objective_function_value_against_iteration.png')  # Save plot to file

# Example function not directly used in the main workflow, demonstrating a utility function
def parity(x):
    """Example function to calculate parity of an integer."""
    return "{:b}".format(x).count("1") % 2

# Initializing the quantum sampler from Qiskit
from qiskit.primitives import Sampler
sampler = Sampler()  # Used for sampling from quantum circuits

# Constructing the quantum feature map and ansatz for the quantum circuit
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes
feature_map = ZZFeatureMap(feature_dimension)
ansatz = RealAmplitudes(feature_dimension, reps=reps)  # Quantum circuit ansatz

# Composing the quantum circuit with the feature map and ansatz
from qiskit import QuantumCircuit
qc = QuantumCircuit(feature_dimension)
qc.append(feature_map, range(feature_dimension))  # Apply feature map to circuit
qc.append(ansatz, range(feature_dimension))  # Apply ansatz to circuit
qc.decompose().draw()  # Draw and decompose circuit for visualization

# Creating a Quantum Neural Network (QNN) using the configured quantum circuit
from qiskit_machine_learning.neural_networks import SamplerQNN
sampler_qnn = SamplerQNN(
    circuit=qc,
    input_params=feature_map.parameters,
    weight_params=ansatz.parameters,
    output_shape=2,  # For binary classification
    sampler=sampler
)

# Configuring the quantum classifier with the COBYLA optimizer
from qiskit.algorithms.optimizers import COBYLA
from qiskit_machine_learning.algorithms.classifiers import NeuralNetworkClassifier
sampler_classifier = NeuralNetworkClassifier(
    neural_network=sampler_qnn, optimizer=COBYLA(maxiter=100), callback=callback_graph)

# Setting up K-Fold Cross Validation to assess model performance
from sklearn.model_selection import KFold
k_fold = KFold(n_splits=5)  # 5-fold cross-validation
score = np.zeros(5)  # Array to store scores for each fold
i = 0  # Index counter for scores array
for indices_train, indices_test in k_fold.split(X):
    X_train, X_test = X.iloc[indices_train], X.iloc[indices_test]
    y_train, y_test = y[indices_train], y[indices_test]
    
    # Applying PCA to reduce the dimensionality of the dataset to match the quantum feature map
    from sklearn.decomposition import PCA
    pca = PCA(n_components=2)  # Reduce to 2 dimensions for the quantum circuit
    X_train = pca.fit_transform(X_train)  # Transform training set
    X_test = pca.fit_transform(X_test)  # Transform test set
    
    # Training the quantum classifier with the training set
    sampler_classifier.fit(X_train, y_train)
    
    # Evaluating the classifier's performance on the test set
    score[i] = sampler_classifier.score(X_test, y_test)  # Store score for this fold
    i += 1  # Increment index for next score

# Calculating and displaying the results of cross-validation
import math
print("Cross-validation scores:", score)
cross_mean = np.mean(score)  # Mean of cross-validation scores
cross_var = np.var(score)  # Variance of scores
cross_std = math.sqrt(cross_var)  # Standard deviation of scores

print("Mean cross-validation score:", cross_mean)
print("Standard deviation of cross-validation scores:", cross_std)

We obtain the following results:

Cross-validation scores: [0.34210526 0.4122807 0.42982456 0.21929825 0.50442478]

Mean cross-validation score: 0.3815867101381773

Standard deviation of cross-validation scores: 0.09618163326986424

As we can see, in this specific dataset, QNN doesn’t provide a very good classification score.

Conclusion

This idea of this blog is to make it easy to start using quantum machine learning. Quantum Machine Learning is an emerging field at the intersection of quantum computing and machine learning that holds the potential to revolutionize how we process and analyze vast datasets by leveraging the inherent advantages of quantum mechanics. As we showed in our paper Application of quantum machine learning using quantum kernel algorithms on multiclass neuron M-type classification published in Nature Scientific Report, a crucial aspect of optimizing QML models, including Quantum Neural Networks (QNNs), involves pre-processing techniques such as feature rescaling, feature extraction, and feature selection.

These techniques are not only essential in classical machine learning but also present significant benefits when applied within the quantum computing framework, enhancing the performance and efficiency of quantum machine learning algorithms. In the quantum realm, feature extraction techniques like Principal Component Analysis (PCA) can be quantum-enhanced to reduce the dimensionality of the data while retaining most of its significant information. This reduction is vital for QML models due to the limited number of qubits available on current quantum hardware.

Quantum feature extraction can efficiently map high-dimensional data into a lower-dimensional quantum space, enabling quantum models to process complex datasets with fewer resources. Selecting the most relevant features is also a way for optimizing quantum circuit complexity and resource allocation. In quantum machine learning, feature selection helps in identifying and utilizing the most informative features, reducing the need for extensive quantum resources.

This process not only simplifies the quantum models but also enhances their performance by focusing the computational efforts on the features that contribute the most to the predictive accuracy of the model.

Sources

Machine Learning Theory and Applications: Hands-on Use Cases with Python on Classical and Quantum Machines, Wiley, 2024

Vasques, X., Paik, H. & Cif, L. Application of quantum machine learning using quantum kernel algorithms on multiclass neuron M-type classification. Sci Rep 13, 11541 (2023). https://doi.org/10.1038/s41598-023-38558-z

This dataset used is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Quantum Machine Learning with Python: Kernel Methods and Neural Networks was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.