Databricks Runtime 16.0 per Machine Learning

Databricks Runtime 16.0 per Machine Learning offre un ambiente pronto per l'apprendimento automatico e l'analisi scientifica dei dati basato su Databricks Runtime 16.0. Databricks Runtime per Machine Learning contiene molte di queste librerie, tra cui TensorFlow, PyTorch, Keras e XGBoost. Databricks Runtime ML include AutoML, uno strumento per eseguire automaticamente il training delle pipeline di Machine Learning. Databricks Runtime ML supporta anche il training di Deep Learning distribuito usando TorchDistributor, DeepSpeed e Ray.

Suggerimento

Per visualizzare le note sulla versione per le versioni di Databricks Runtime che hanno raggiunto la fine del supporto (EoS), vedere Note sulla versione della fine del supporto di Databricks Runtime. Le versioni EoS di Databricks Runtime sono state ritirate e potrebbero non essere aggiornate.

Miglioramenti e nuove funzionalità

Databricks Runtime 16.0 ML è basato su Databricks Runtime 16.0. Per informazioni sulle novità di Databricks Runtime 16.0, tra cui Apache Spark MLlib e SparkR, vedere le note sulla versione di Databricks Runtime 16.0 .

Nuovi pacchetti Python

I pacchetti Python seguenti sono stati aggiunti a Databricks Runtime ML:

  • composer 0.24.1
  • optuna 3.6.1

Pesi del campione di Mosaic AutoML per la previsione

AutoML supporta ora i pesi di esempio per la previsione, consentendo di regolare l'importanza di ogni serie temporale per il training di modelli di previsione di serie multi-time. Per altre informazioni, vedere i parametri di previsione per l'API Python AutoML.

Usare una vista in Unity Catalog come tabella delle funzionalità

È ora possibile usare una vista in Unity Catalog come tabella delle funzionalità. Vedere Usare una vista esistente in Unity Catalog come tabella delle funzionalità.

Altre modifiche

Horovod, HorovodRunner, Petastorm, spark-tensorflow-distributor rimosso

I pacchetti seguenti inclusi nelle versioni precedenti di Databricks Runtime ML non sono inclusi in Databricks Runtime 16.0 ML:

  • Horovod
  • HorovodRunner
  • Petastorm
  • spark-tensorflow-distributor

Databricks consiglia le sostituzioni seguenti:

  • Per l'apprendimento avanzato distribuito, Databricks consiglia di utilizzare TorchDistributor per il training distribuito con PyTorch o l'API tf.distribute.Strategy per il training distribuito con TensorFlow.
  • Per caricare set di dati di grandi dimensioni dall'archiviazione cloud, Databricks consiglia di usare Mosaic Streaming.
  • Per il training distribuito per un modello TensorFlow o Keras, Databricks consiglia di usare Ray. Vedere Ray on Databricks e la documentazione di Ray.

Ambiente di sistema

L'ambiente di sistema in Databricks Runtime 16.0 ML differisce da Databricks Runtime 16.0 come indicato di seguito:

  • Per i cluster GPU, Databricks Runtime ML include le librerie GPU NVIDIA seguenti:
    • CUDA 12.6
    • cublas 12.6.0.22-1
    • cusolver 11.6.4.38-1
    • cupti 12.6.37-1
    • cusparse 12.5.2.23-1
    • cuDNN 9.3.0.75-1
    • NCCL 2.22.3
    • TensorRT 10.2.0.19-1

Librerie

Le sezioni seguenti elencano le librerie incluse in Databricks Runtime 16.0 ML che differiscono da quelle incluse in Databricks Runtime 16.0.

Contenuto della sezione:

Librerie di livello superiore

Databricks Runtime 16.0 ML include le librerie di livello superiore seguenti:

Librerie Python

Databricks Runtime 16.0 ML usa virtualenv per la gestione dei pacchetti Python e include molti pacchetti di Machine Learning più diffusi.

Oltre ai pacchetti specificati nelle sezioni seguenti, Databricks Runtime 16.0 ML include anche i pacchetti seguenti:

  • hyperopt 0.2.7+db5
  • automl 1.29.0

Per riprodurre l'ambiente Python di Databricks Runtime ML nell'ambiente virtuale Python locale, scaricare il file di requirements-16.0.txt ed eseguire pip install -r requirements-16.0.txt. Questo comando installa tutte le librerie open source usate da Databricks Runtime ML, ma non installa librerie sviluppate da Databricks, ad esempio databricks-automl, databricks-feature-engineering o il fork di Databricks di hyperopt.

Librerie Python nei cluster CPU

Library Versione Library Versione Library Versione
absl-py 1.0.0 accelerate 0.33.0 aiohttp 3.9.5
aiohttp-cors 0.7.0 aiosignal 1.2.0 Alembic 1.13.3
tipi annotati 0.7.0 anyio 4.2.0 argcomplete 3.5.0
argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 freccia 1.2.3
astor 0.8.1 asttoken 2.0.5 astunparse 1.6.3
async-lru 2.0.4 attrs 23.1.0 audioread 3.0.1
comando automatico 2.2.2 azure-core 1.31.0 azure-cosmos 4.3.1
azure-identity 1.18.0 azure-storage-blob 12.23.1 azure-storage-file-datalake 12.17.0
Babel 2.11.0 backoff 2.2.1 backports.tarfile 1.2.0
bcrypt 3.2.0 beautifulsoup4 4.12.3 black 24.4.2
bleach 4.1.0 blinker 1.7.0 blis 0.7.11
boto3 1.34.69 botocore 1.34.69 Brotli 1.0.9
cachetools 5.3.3 catalogue 2.0.10 category-encoders 2.6.3
certifi 2024.6.2 cffi 1.16.0 chardet 4.0.0
charset-normalizer 2.0.4 circuitbreaker 2.0.0 click 8.1.7
cloudpathlib 0.19.0 cloudpickle 2.2.1 cmdstanpy 1.2.4
a colori 0.5.6 colorlog 6.8.2 serv 0.2.1
composer 0.24.1 confection 0.1.5 configparser 5.2.0
contourpy 1.2.0 coolname 2.2.0 cryptography 42.0.5
cycler 0.11.0 cymem 2.0.8 Cython 3.0.11
dacite 1.8.1 databricks-automl-runtime 0.2.21 databricks-feature-engineering 0.7.0
databricks-sdk 0.30.0 datasets 2.20.0 dbl-tempo 0.1.26
dbus-python 1.3.2 debugpy 1.6.7 decorator 5.1.1
deepspeed 0.14.4 defusedxml 0.7.1 Deprecato 1.2.14
dill 0.3.8 distlib 0.3.8 dm-tree 0.1.8
docstring-to-markdown 0.11 entrypoints 0.4 evaluate 0.4.2
executing 0.8.3 facet-overview 1.1.1 Farama-Notifications 0.0.4
fastjsonschema 2.20.0 fasttext-wheel 0.9.2 filelock 3.13.1
Flask 2.2.5 flatbuffers 24.3.25 fonttools 4.51.0
fqdn 1.5.1 frozenlist 1.4.0 fsspec 2023.5.0
future 0.18.3 gast 0.4.0 gitdb 4.0.11
GitPython 3.1.37 google-api-core 2.20.0 google-auth 2.21.0
google-auth-oauthlib 1.0.0 google-cloud-core 2.4.1 google-cloud-storage 2.10.0
google-crc32c 1.6.0 google-pasta 0.2.0 google-resumable-media 2.7.2
googleapis-common-protos 1.65.0 gql 3.5.0 graphql-core 3.2.4
greenlet 3.0.1 grpcio 1.60.0 grpcio-status 1.60.0
gunicorn 20.1.0 gviz-api 1.10.0 gymnasium 0.28.1
h11 0.14.0 h5py 3.11.0 hjson 3.1.0
holidays 0,54 htmlmin 0.1.12 httpcore 1.0.5
httplib2 0.20.4 httpx 0.27.2 huggingface-hub 0.24.5
idna 3.7 ImageHash 4.3.1 imageio 2.33.1
sbilanciato-learn 0.12.3 importlib-metadata 6.0.0 importlib_resources 6.4.5
flettere 7.3.1 ipyflow-core 0.0.201 ipykernel 6.28.0
ipython 8.25.0 ipython-genutils 0.2.0 ipywidgets 7.7.2
isodate 0.6.1 isoduration 20.11.0 itsdangerous 2.2.0
jaraco.context 5.3.0 jaraco.functools 4.0.1 jaraco.text 3.12.1
jax jumpy 1.0.0 jedi 0.19.1 Jinja2 3.1.4
jiter 0.5.0 jmespath 1.0.1 joblib 1.4.2
joblibspark 0.5.1 json5 0.9.6 jsonpatch 1.33
jsonpointer 3.0.0 jsonschema 4.19.2 jsonschema-specifications 2023.7.1
jupyter-events 0.10.0 jupyter-lsp 2.2.0 jupyter_client 8.6.0
jupyter_core 5.7.2 jupyter_server 2.14.1 jupyter_server_terminals 0.4.4
jupyterlab 4.0.11 jupyterlab-pygments 0.1.2 jupyterlab_server 2.25.1
keras 3.5.0 kiwisolver 1.4.4 langchain 0.2.12
langchain-core 0.2.41 langchain-text-splitters 0.2.4 langcodes 3.4.1
langsmith 0.1.129 language_data 1.2.0 launchpadlib 1.11.0
lazr.restfulclient 0.14.6 lazr.uri 1.0.6 lazy_loader 0.4
libclang 15.0.6.1 librosa 0.10.2 lightgbm 4.5.0
fulmine-utilità 0.11.7 linkify-it-py 2.0.0 llvmlite 0.42.0
lz4 4.3.2 Mako 1.2.0 marisa-trie 1.2.0
Markdown 3.4.1 markdown-it-py 2.2.0 MarkupSafe 2.1.3
matplotlib 3.8.4 matplotlib-inline 0.1.6 mccabe 0.7.0
mdit-py-plugins 0.3.0 mdurl 0.1.0 memray 1.14.0
mistune 2.0.4 ml-dtypes 0.4.1 mlflow-skinny 2.15.1
more-itertools 10.3.0 mosaicml-cli 0.6.41 mosaicml-streaming 0.8.0
mpmath 1.3.0 msal 1.31.0 msal-extensions 1.2.0
msgpack 1.1.0 multidict 6.0.4 multimethod 1.12
multiprocess 0.70.16 murmurhash 1.0.10 mypy 1.10.0
mypy-extensions 1.0.0 namex 0.0.8 nbclient 0.8.0
nbconvert 7.10.0 nbformat 5.9.2 nest-asyncio 1.6.0
networkx 3.2.1 ninja 1.11.1.1 nltk 3.8.1
nodeenv 1.9.1 notebook 7.0.8 notebook_shim 0.2.3
numba 0.59.1 numpy 1.26.4 nvidia-ml-py 12.560.30
oauthlib 3.2.0 oci 2.135.0 openai 1.40.2
opencensus 0.11.4 opencensus-context 0.1.3 opentelemetry-api 1.27.0
opentelemetry-sdk 1.27.0 opentelemetry-semantic-conventions 0.48b0 opt_einsum 3.4.0
optree 0.12.1 optuna 3.6.1 optuna-integration 3.6.0
orjson 3.10.7 override 7.4.0 packaging 24.1
pandas 1.5.3 pandocfilters 1.5.0 paramiko 3.4.0
parso 0.8.3 pathspec 0.10.3 patsy 0.5.6
pexpect 4.8.0 phik 0.12.4 pillow 10.3.0
pip 24.2 platformdirs 3.10.0 plotly 5.22.0
pluggy 1.0.0 pmdarima 2.0.4 pooch 1.8.2
portalocker 2.10.1 preshed 3.0.9 prometheus-client 0.14.1
prompt-toolkit 3.0.43 prophet 1.1.5 proto-plus 1.24.0
protobuf 4.24.1 psutil 5.9.0 psycopg2 2.9.3
ptyprocess 0.7.0 pure-eval 0.2.2 py-cpuinfo 9.0.0
py-spy 0.3.14 pyarrow 15.0.2 pyarrow-hotfix 0,6
pyasn1 0.4.8 pyasn1-modules 0.2.8 PyBind11 2.13.6
pyccolo 0.0.65 pycparser 2.21 pydantic 2.8.2
pydantic_core 2.20.1 pyflakes 3.2.0 Pygments 2.15.1
PyGObject 3.48.2 PyJWT 2.7.0 PyNaCl 1.5.0
pyodbc 5.0.1 pyOpenSSL 24.0.0 pyparsing 3.0.9
pyright 1.1.294 pytesseract 0.3.10 python-dateutil 2.9.0.post0
python-editor 1.0.4 python-json-logger 2.0.7 python-lsp-jsonrpc 1.1.2
python-lsp-server 1.10.0 python-snappy 0.6.1 pytoolconfig 1.2.6
pytorch-ranger 0.1.1 pytz 2024.1 PyWavelets 1.5.0
PyYAML 6.0.1 pyzmq 25.1.2 interrogatorio 1.10.0
ray 2.35.0 referencing 0.30.2 regex 2023.10.3
requests 2.32.2 requests-oauthlib 1.3.1 rfc3339-validator 0.1.4
rfc3986-validator 0.1.1 rich 13.3.5 rope 1.12.0
rpds-py 0.10.6 rsa 4.9 michelemel.yaml 0.18.6
michelemel.yaml.clib 0.2.8 s3transfer 0.10.2 safetensors 0.4.4
scikit-image 0.23.2 scikit-learn 1.4.2 scipy 1.13.1
seaborn 0.13.2 Send2Trash 1.8.2 sentence-transformers 3.0.1
sentencepiece 0.2.0 setuptools 74.0.0 shap 0.46.0
shellingham 1.5.4 simplejson 3.17.6 six 1.16.0
slicer 0.0.8 smart-open 5.2.1 smmap 5.0.0
sniffio 1.3.0 soundfile 0.12.1 soupsieve 2.5
soxr 0.5.0.post1 spaCy 3.7.5 spacy-legacy 3.0.12
spacy-logger 1.0.5 SQLAlchemy 2.0.30 sqlparse 0.4.2
srsly 2.4.8 ssh-import-id 5.11 stack-data 0.2.0
stanio 0.5.1 statsmodels 0.14.2 sympy 1.12
tabulate 0.9.0 tangled-up-in-unicode 0.2.0 tenacity 8.2.2
tensorboard 2.17.0 tensorboard-data-server 0.7.2 tensorboard-plugin-profile 2.17.0
tensorboardX 2.6.2.2 tensorflow 2.17.0 tensorflow-estimator 2.15.0
termcolor 2.4.0 terminado 0.17.1 textual 0.81.0
tf_keras 2.17.0 thinc 8.2.5 threadpoolctl 2.2.0
tifffile 2023.4.12 tiktoken 0.7.0 tinycss2 1.2.1
tokenize-rt 4.2.1 tokenizers 0.19.1 tomli 2.0.1
torch 2.4.0+CPU torch-optimizer 0.3.0 torcheval 0.0.7
torchmetrics 1.4.0.post0 torchvision 0.19.0+CPU tornado 6.4.1
tqdm 4.66.4 traitlets 5.14.3 Convertitori 4.44.0
typeguard 4.3.0 Typer 0.12.5 types-protobuf 3.20.3
types-psutil 5.9.0 types-pytz 2023.3.1.1 types-PyYAML 6.0.0
richieste di tipi 2.31.0.0 types-setuptools 68.0.0.0 tipi-sei 1.16.0
types-urllib3 1.26.25.14 typing_extensions 4.11.0 uc-micro-py 1.0.1
ujson 5.10.0 aggiornamenti automatici 0.1 Modello URI 1.3.0
urllib3 1.26.16 validator 0.34.0 virtualenv 20.26.2
visions 0.7.5 wadllib 1.3.6 wasabi 1.1.3
wcwidth 0.2.5 weasel 0.4.1 webcolors 24.8.0
webencodings 0.5.1 websocket-client 1.8.0 websockets 11.0.3
Werkzeug 3.0.3 whatthepatch 1.0.2 wheel 0.43.0
wordcloud 1.9.3 wrapt 1.14.1 xgboost 2.0.3
xgboost-ray 0.1.19 xxhash 3.4.1 yapf 0.33.0
yarl 1.9.3 ydata-profiling 4.9.0 zipp 3.17.0
zstd 1.5.5.1

Librerie Python nei cluster GPU

Library Versione Library Versione Library Versione
absl-py 1.0.0 accelerate 0.33.0 aiohttp 3.9.5
aiohttp-cors 0.7.0 aiosignal 1.2.0 tipi annotati 0.7.0
anyio 4.2.0 argcomplete 3.5.0 argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0 freccia 1.2.3 astor 0.8.1
asttoken 2.0.5 astunparse 1.6.3 async-lru 2.0.4
attrs 23.1.0 audioread 3.0.1 comando automatico 2.2.2
azure-core 1.31.0 azure-cosmos 4.3.1 azure-identity 1.18.0
azure-storage-blob 12.23.1 azure-storage-file-datalake 12.17.0 Babel 2.11.0
backoff 2.2.1 backports.tarfile 1.2.0 bcrypt 3.2.0
beautifulsoup4 4.12.3 black 24.4.2 bleach 4.1.0
blinker 1.7.0 blis 0.7.11 boto3 1.34.69
botocore 1.34.69 Brotli 1.0.9 cachetools 5.3.3
catalogue 2.0.10 category-encoders 2.6.3 certifi 2024.6.2
cffi 1.16.0 chardet 4.0.0 charset-normalizer 2.0.4
circuitbreaker 2.0.0 click 8.1.7 cloudpathlib 0.19.0
cloudpickle 2.2.1 cmdstanpy 1.2.4 a colori 0.5.6
colorlog 6.8.2 serv 0.2.1 composer 0.24.1
confection 0.1.5 configparser 5.2.0 contourpy 1.2.0
coolname 2.2.0 cryptography 42.0.5 cycler 0.11.0
cymem 2.0.8 Cython 3.0.11 dacite 1.8.1
databricks-automl-runtime 0.2.21 databricks-feature-engineering 0.7.0 databricks-sdk 0.30.0
datasets 2.20.0 dbl-tempo 0.1.26 dbus-python 1.3.2
debugpy 1.6.7 decorator 5.1.1 deepspeed 0.14.4
defusedxml 0.7.1 Deprecato 1.2.14 dill 0.3.8
distlib 0.3.8 dm-tree 0.1.8 docstring-to-markdown 0.11
einops 0.8.0 entrypoints 0.4 evaluate 0.4.2
executing 0.8.3 facet-overview 1.1.1 Farama-Notifications 0.0.4
fastjsonschema 2.20.0 fasttext-wheel 0.9.2 filelock 3.13.1
flash_attn 2.5.6 Flask 2.2.5 flatbuffers 24.3.25
fonttools 4.51.0 fqdn 1.5.1 frozenlist 1.4.0
fsspec 2023.5.0 future 0.18.3 gast 0.4.0
gitdb 4.0.11 GitPython 3.1.37 google-api-core 2.20.0
google-auth 2.21.0 google-auth-oauthlib 1.0.0 google-cloud-core 2.4.1
google-cloud-storage 2.10.0 google-crc32c 1.6.0 google-pasta 0.2.0
google-resumable-media 2.7.2 googleapis-common-protos 1.65.0 gql 3.5.0
graphql-core 3.2.4 greenlet 3.0.1 grpcio 1.60.0
grpcio-status 1.60.0 gunicorn 20.1.0 gviz-api 1.10.0
gymnasium 0.28.1 h11 0.14.0 h5py 3.11.0
hjson 3.1.0 holidays 0,54 htmlmin 0.1.12
httpcore 1.0.5 httplib2 0.20.4 httpx 0.27.2
huggingface-hub 0.24.5 idna 3.7 ImageHash 4.3.1
imageio 2.33.1 sbilanciato-learn 0.12.3 importlib-metadata 6.0.0
importlib_resources 6.4.5 flettere 7.3.1 ipyflow-core 0.0.201
ipykernel 6.28.0 ipython 8.25.0 ipython-genutils 0.2.0
ipywidgets 7.7.2 isodate 0.6.1 isoduration 20.11.0
itsdangerous 2.2.0 jaraco.context 5.3.0 jaraco.functools 4.0.1
jaraco.text 3.12.1 jax jumpy 1.0.0 jedi 0.19.1
Jinja2 3.1.4 jiter 0.5.0 jmespath 1.0.1
joblib 1.4.2 joblibspark 0.5.1 json5 0.9.6
jsonpatch 1.33 jsonpointer 3.0.0 jsonschema 4.19.2
jsonschema-specifications 2023.7.1 jupyter-events 0.10.0 jupyter-lsp 2.2.0
jupyter_client 8.6.0 jupyter_core 5.7.2 jupyter_server 2.14.1
jupyter_server_terminals 0.4.4 jupyterlab 4.0.11 jupyterlab-pygments 0.1.2
jupyterlab_server 2.25.1 keras 3.5.0 kiwisolver 1.4.4
langchain 0.2.12 langchain-core 0.2.41 langchain-text-splitters 0.2.4
langcodes 3.4.1 langsmith 0.1.129 language_data 1.2.0
launchpadlib 1.11.0 lazr.restfulclient 0.14.6 lazr.uri 1.0.6
lazy_loader 0.4 libclang 15.0.6.1 librosa 0.10.2
lightgbm 4.5.0 fulmine-utilità 0.11.7 linkify-it-py 2.0.0
llvmlite 0.42.0 lz4 4.3.2 Mako 1.2.0
marisa-trie 1.2.0 Markdown 3.4.1 markdown-it-py 2.2.0
MarkupSafe 2.1.3 matplotlib 3.8.4 matplotlib-inline 0.1.6
mccabe 0.7.0 mdit-py-plugins 0.3.0 mdurl 0.1.0
memray 1.14.0 mistune 2.0.4 ml-dtypes 0.4.1
mlflow-skinny 2.15.1 more-itertools 10.3.0 mosaicml-cli 0.6.41
mosaicml-streaming 0.8.0 mpmath 1.3.0 msal 1.31.0
msal-extensions 1.2.0 msgpack 1.1.0 multidict 6.0.4
multimethod 1.12 multiprocess 0.70.16 murmurhash 1.0.10
mypy 1.10.0 mypy-extensions 1.0.0 namex 0.0.8
nbclient 0.8.0 nbconvert 7.10.0 nbformat 5.9.2
nest-asyncio 1.6.0 networkx 3.2.1 ninja 1.11.1.1
nltk 3.8.1 nodeenv 1.9.1 notebook 7.0.8
notebook_shim 0.2.3 numba 0.59.1 numpy 1.26.4
nvidia-cublas-cu12 12.4.2.65 nvidia-cuda-cupti-cu12 12.4.99 nvidia-cuda-nvrtc-cu12 12.4.99
nvidia-cuda-runtime-cu12 12.4.99 nvidia-cudnn-cu12 9.1.0.70 nvidia-cufft-cu12 11.2.0.44
nvidia-curand-cu12 10.3.5.119 nvidia-cusolver-cu12 11.6.0.99 nvidia-cusparse-cu12 12.3.0.142
nvidia-ml-py 12.560.30 nvidia-nccl-cu12 2.20.5 nvidia-nvjitlink-cu12 12.4.99
nvidia-nvtx-cu12 12.4.99 oauthlib 3.2.0 oci 2.135.0
openai 1.40.2 opencensus 0.11.4 opencensus-context 0.1.3
opentelemetry-api 1.27.0 opentelemetry-sdk 1.27.0 opentelemetry-semantic-conventions 0.48b0
opt_einsum 3.4.0 optree 0.12.1 optuna 3.6.1
optuna-integration 3.6.0 orjson 3.10.7 override 7.4.0
packaging 24.1 pandas 1.5.3 pandocfilters 1.5.0
paramiko 3.4.0 parso 0.8.3 pathspec 0.10.3
patsy 0.5.6 pexpect 4.8.0 phik 0.12.4
pillow 10.3.0 pip 24.2 platformdirs 3.10.0
plotly 5.22.0 pluggy 1.0.0 pmdarima 2.0.4
pooch 1.8.2 portalocker 2.10.1 preshed 3.0.9
prometheus-client 0.14.1 prompt-toolkit 3.0.43 prophet 1.1.5
proto-plus 1.24.0 protobuf 4.24.1 psutil 5.9.0
psycopg2 2.9.3 ptyprocess 0.7.0 pure-eval 0.2.2
py-cpuinfo 9.0.0 py-spy 0.3.14 pyarrow 15.0.2
pyarrow-hotfix 0,6 pyasn1 0.4.8 pyasn1-modules 0.2.8
PyBind11 2.13.6 pyccolo 0.0.65 pycparser 2.21
pydantic 2.8.2 pydantic_core 2.20.1 pyflakes 3.2.0
Pygments 2.15.1 PyGObject 3.48.2 PyJWT 2.7.0
PyNaCl 1.5.0 pyodbc 5.0.1 pyOpenSSL 24.0.0
pyparsing 3.0.9 pyright 1.1.294 pytesseract 0.3.10
python-dateutil 2.9.0.post0 python-editor 1.0.4 python-json-logger 2.0.7
python-lsp-jsonrpc 1.1.2 python-lsp-server 1.10.0 python-snappy 0.6.1
pytoolconfig 1.2.6 pytorch-ranger 0.1.1 pytz 2024.1
PyWavelets 1.5.0 PyYAML 6.0.1 pyzmq 25.1.2
interrogatorio 1.10.0 ray 2.35.0 referencing 0.30.2
regex 2023.10.3 requests 2.32.2 requests-oauthlib 1.3.1
rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.3.5
rope 1.12.0 rpds-py 0.10.6 rsa 4.9
michelemel.yaml 0.18.6 michelemel.yaml.clib 0.2.8 s3transfer 0.10.2
safetensors 0.4.4 scikit-image 0.23.2 scikit-learn 1.4.2
scipy 1.13.1 seaborn 0.13.2 Send2Trash 1.8.2
sentence-transformers 3.0.1 sentencepiece 0.2.0 setuptools 74.0.0
shap 0.46.0 shellingham 1.5.4 simplejson 3.17.6
six 1.16.0 slicer 0.0.8 smart-open 5.2.1
smmap 5.0.0 sniffio 1.3.0 soundfile 0.12.1
soupsieve 2.5 soxr 0.5.0.post1 spaCy 3.7.5
spacy-legacy 3.0.12 spacy-logger 1.0.5 SQLAlchemy 2.0.30
sqlparse 0.4.2 srsly 2.4.8 ssh-import-id 5.11
stack-data 0.2.0 stanio 0.5.1 statsmodels 0.14.2
sympy 1.12 tabulate 0.9.0 tangled-up-in-unicode 0.2.0
tenacity 8.2.2 tensorboard 2.17.0 tensorboard-data-server 0.7.2
tensorboard-plugin-profile 2.17.0 tensorboardX 2.6.2.2 tensorflow 2.17.0
tensorflow-estimator 2.15.0 termcolor 2.4.0 terminado 0.17.1
textual 0.81.0 tf_keras 2.17.0 thinc 8.2.5
threadpoolctl 2.2.0 tifffile 2023.4.12 tiktoken 0.7.0
tinycss2 1.2.1 tokenize-rt 4.2.1 tokenizers 0.19.1
tomli 2.0.1 torch 2.4.0+cu124 torch-optimizer 0.3.0
torcheval 0.0.7 torchmetrics 1.4.0.post0 torchvision 0.19.0+cu124
tornado 6.4.1 tqdm 4.66.4 traitlets 5.14.3
Convertitori 4.44.0 triton 3.0.0 typeguard 4.3.0
Typer 0.12.5 types-protobuf 3.20.3 types-psutil 5.9.0
types-pytz 2023.3.1.1 types-PyYAML 6.0.0 richieste di tipi 2.31.0.0
types-setuptools 68.0.0.0 tipi-sei 1.16.0 types-urllib3 1.26.25.14
typing_extensions 4.11.0 uc-micro-py 1.0.1 ujson 5.10.0
aggiornamenti automatici 0.1 Modello URI 1.3.0 urllib3 1.26.16
validator 0.34.0 virtualenv 20.26.2 visions 0.7.5
wadllib 1.3.6 wasabi 1.1.3 wcwidth 0.2.5
weasel 0.4.1 webcolors 24.8.0 webencodings 0.5.1
websocket-client 1.8.0 websockets 11.0.3 Werkzeug 3.0.3
whatthepatch 1.0.2 wheel 0.43.0 wordcloud 1.9.3
wrapt 1.14.1 xgboost 2.0.3 xgboost-ray 0.1.19
xxhash 3.4.1 yapf 0.33.0 yarl 1.9.3
ydata-profiling 4.9.0 zipp 3.17.0 zstd 1.5.5.1

Librerie R

Le librerie R sono identiche alle librerie R in Databricks Runtime 16.0.

Librerie Java e Scala (cluster Scala 2.12)

Oltre alle librerie Java e Scala in Databricks Runtime 16.0, Databricks Runtime 16.0 ML contiene i file JAR seguenti:

Cluster CPU

ID gruppo ID artefatto Versione
com.typesafe.akka akka-actor_2.12 2.5.23
ml.dmlc xgboost4j-spark_2.12 1.7.3
ml.dmlc xgboost4j_2.12 1.7.3
org.graphframes graphframes_2.12 0.8.4-db1-spark3.5
org.mlflow mlflow-client 2.15.1
org.tensorflow spark-tensorflow-connector_2.12 1.15.0

Cluster GPU

ID gruppo ID artefatto Versione
com.typesafe.akka akka-actor_2.12 2.5.23
ml.dmlc xgboost4j-gpu_2.12 1.7.3
ml.dmlc xgboost4j-spark-gpu_2.12 1.7.3
org.graphframes graphframes_2.12 0.8.4-db1-spark3.5
org.mlflow mlflow-client 2.15.1
org.tensorflow spark-tensorflow-connector_2.12 1.15.0