{"id":58,"date":"2014-09-17T17:48:57","date_gmt":"2014-09-17T15:48:57","guid":{"rendered":"https:\/\/www.cubeos.org\/blog\/2014\/09\/linux-and-azure-files-you-might-need-some-help-here\/"},"modified":"2014-09-17T17:48:57","modified_gmt":"2014-09-17T15:48:57","slug":"linux-and-azure-files-you-might-need-some-help-here","status":"publish","type":"post","link":"https:\/\/www.cubeos.org\/blog\/2014\/09\/linux-and-azure-files-you-might-need-some-help-here\/","title":{"rendered":"Linux and Azure Files: you might need some help here\u2026"},"content":{"rendered":"<p>&#160;<\/p>\n<p>Hi!<\/p>\n<p>tl;dr: To mount Azure Files from linux, you need cifs support in the kernel, the right mount helper and versions recent enough to supports the SMB2 protocol version.<\/p>\n<p>I just got a ping from a customer who had trouble mounting an Azure Files filesystem from Linux. According to the Azure team blog, this should work: <a title=\"http:\/\/blogs.msdn.com\/b\/windowsazurestorage\/archive\/2014\/05\/12\/introducing-microsoft-azure-file-service.aspx\" href=\"http:\/\/blogs.msdn.com\/b\/windowsazurestorage\/archive\/2014\/05\/12\/introducing-microsoft-azure-file-service.aspx\" target=\"_blank\" rel=\"nofollow\">http:\/\/blogs.msdn.com\/b\/windowsazurestorage\/archive\/2014\/05\/12\/introducing-microsoft-azure-file-service.aspx<\/a>&#160;<\/p>\n<p>So I tried it myself on a Ubuntu 14.04 LTS and found the following:<\/p>\n<p>If I used smbclient, everything worked:<\/p>\n<p>kenn@cubefileclient:~$ smbclient -d 3 \/\/cubefiles.file.core.windows.net\/cubefiletest &lt;storage key goes here&gt; -U cubefiles -m SMB2    <br \/>[lots of debug output deleted here]     <br \/>Connecting to 168.61.61.18 at port 445     <br \/>Doing spnego session setup (blob length=0)     <br \/>server didn&#8217;t supply a full spnego negprot     <br \/>Got challenge flags:     <br \/>Got NTLMSSP neg_flags=0x628a8015     <br \/>NTLMSSP: Set final flags:     <br \/>Got NTLMSSP neg_flags=0x60088215     <br \/>NTLMSSP Sign\/Seal &#8211; Initialising with flags:     <br \/>Got NTLMSSP neg_flags=0x60088215     <br \/>Domain=[X] OS=[] Server=[]     <br \/>smb: &gt; dir<\/p>\n<p>&#160; .&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; D&#160;&#160;&#160;&#160;&#160;&#160;&#160; 0&#160; Mon Sep&#160; 8 14:49:55 2014    <br \/>&#160; ..&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; D&#160;&#160;&#160;&#160;&#160;&#160;&#160; 0&#160; Mon Sep&#160; 8 14:49:55 2014     <br \/>&#160; testdir&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; D&#160;&#160;&#160;&#160;&#160;&#160;&#160; 0&#160; Mon Sep&#160; 8 14:47:08 2014     <br \/>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 83886080 blocks of size 65536. 83886080 blocks available     <br \/>Total bytes listed: 0     <br \/>smb: &gt; quit<\/p>\n<p>Don\u2019t be alarmed by all those scary looking messages, I\u2019m running smbclient with \u2013d 3, so there are a lot of debug messages.<\/p>\n<p>Now I tried to mount the filesystem:<\/p>\n<p>kenn@cubefileclient:~$ sudo bash    <br \/>root@cubefileclient:~# mount -t cifs <a href=\"\/\/\/\\cubefiles.file.core.windows.netcubefiletest\" target=\"_blank\" rel=\"nofollow\">\\cubefiles.file.core.windows.netcubefiletest<\/a> \/mountpoint -o vers=2.1,username=cubefiles,password=&lt;storage key goes here&gt;,dir_mode=0777,file_mode=0777     <br \/>mount: wrong fs type, bad option, bad superblock on <a href=\"\/\/cubefiles.file.core.windows.netcubefiletest\" target=\"_blank\" rel=\"nofollow\">cubefiles.file.core.windows.netcubefiletest<\/a>,     <br \/>&#160;&#160;&#160;&#160;&#160;&#160; missing codepage or helper program, or other error     <br \/>&#160;&#160;&#160;&#160;&#160;&#160; (for several filesystems (e.g. nfs, cifs) you might     <br \/>&#160;&#160;&#160;&#160;&#160;&#160; need a \/sbin\/mount.&lt;type&gt; helper program)     <br \/>&#160;&#160;&#160;&#160;&#160;&#160; In some cases useful info is found in syslog \u2013 try     <br \/>&#160;&#160;&#160;&#160;&#160;&#160; dmesg | tail&#160; or so<\/p>\n<p>OK, this did not work.<\/p>\n<p>So let\u2019s check if the cifs filesystem is actually in the kernel.<\/p>\n<p>root@cubefileclient:~# grep cifs \/proc\/filesystems    <br \/>nodev&#160;&#160; cifs<\/p>\n<p>Yes, looks good.<\/p>\n<p>So is there a mount helper for cifs?<\/p>\n<p>root@cubefileclient:~# ls -la \/sbin\/mount.cifs    <br \/>ls: cannot access \/sbin\/mount.cifs: No such file or directory<\/p>\n<p>That\u2019s it! we\u2019re missing the mount helper!<\/p>\n<p>root@cubefileclient:~# apt-get install cifs-utils<\/p>\n<p>root@cubefileclient:~# mount -t cifs <a href=\"\/\/\/\\cubefiles.file.core.windows.netcubefiletest\" target=\"_blank\" rel=\"nofollow\">\\cubefiles.file.core.windows.netcubefiletest<\/a> \/mountpoint -o vers=2.1,username=cubefiles,password=&lt;storage key goes here&gt;,dir_mode=0777,file_mode=0777<\/p>\n<p>root@cubefileclient:~# mount    <br \/>[\u2026]     <br \/><a href=\"\/\/\/\\cubefiles.file.core.windows.netcubefiletest\" target=\"_blank\" rel=\"nofollow\">\\cubefiles.file.core.windows.netcubefiletest<\/a> on \/mountpoint type cifs (rw)<\/p>\n<p>root@cubefileclient:~# ls \/mountpoint\/    <br \/>testdir<\/p>\n<p>So this is great, and I thought this was the bug our customer was hitting. But I was wrong: Even with installing the mount helper nothing worked. Even the smbclient did not work for him.<\/p>\n<p>So I recreated his setup (based on Suse Enterprise 11) and I saw the following:<\/p>\n<p>cubefileclient2:~ # smbclient -d 3 \/\/cubefiles.file.core.windows.net\/cubefiletest &lt;storage key goes here&gt; -U cubefiles -m SMB2    <br \/>[lots of debug output deleted here\u2026]     <br \/>protocol negotiation failed: NT_STATUS_PIPE_BROKEN<\/p>\n<p>And also the mount failed.<\/p>\n<p>So I decided to look at what\u2019s going on on the wire. I opened up a second ssh window to the VM and ran tcpdump on the second terminal while attempting to connect to Azure Files in the first. ( tcpdump \u2013s 65535 \u2013w tcpdump.pcap port 445&#160; to be precise)<\/p>\n<p>Since the output of tcpdump wasn\u2019t too enlightening, I decided to load the output using <a href=\"http:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=4865\" target=\"_blank\" rel=\"nofollow\">Microsoft Network Monitor<\/a> and look at the packets there. (To load the capture files from tcpdump, make sure they have the extension .pcap) And then it was quite obvious: <\/p>\n<p>In Ubuntu 14.04 LTS:<\/p>\n<p><a href=\"http:\/\/blogs.msdn.com\/cfs-file.ashx\/__key\/communityserver-blogs-components-weblogfiles\/00-00-01-63-62-metablogapi\/6116.image_5F00_66132FE6.png\" target=\"_blank\" rel=\"nofollow\"><img loading=\"lazy\" decoding=\"async\" title=\"image\" style=\"border-top: 0px;border-right: 0px;border-bottom: 0px;padding-top: 0px;padding-left: 0px;border-left: 0px;padding-right: 0px\" border=\"0\" alt=\"image\" src=\"http:\/\/blogs.msdn.com\/cfs-file.ashx\/__key\/communityserver-blogs-components-weblogfiles\/00-00-01-63-62-metablogapi\/7181.image_5F00_thumb_5F00_4B7C26B8.png\" width=\"360\" height=\"281\" \/><\/a><\/p>\n<p>In Suse Enterprise 11:<\/p>\n<p><a href=\"http:\/\/blogs.msdn.com\/cfs-file.ashx\/__key\/communityserver-blogs-components-weblogfiles\/00-00-01-63-62-metablogapi\/7215.image_5F00_1D780983.png\" target=\"_blank\" rel=\"nofollow\"><img loading=\"lazy\" decoding=\"async\" title=\"image\" style=\"border-top: 0px;border-right: 0px;border-bottom: 0px;padding-top: 0px;padding-left: 0px;border-left: 0px;padding-right: 0px\" border=\"0\" alt=\"image\" src=\"http:\/\/blogs.msdn.com\/cfs-file.ashx\/__key\/communityserver-blogs-components-weblogfiles\/00-00-01-63-62-metablogapi\/2364.image_5F00_thumb_5F00_5B07F7FD.png\" width=\"361\" height=\"243\" \/><\/a><\/p>\n<p>The SMB2 protocol was missing. So I started looking at the version numbers of smbclient, the cifs mount helper and the kernel.<\/p>\n<p>Suse: <\/p>\n<p>cubefileclient2:~ # smbclient -V    <br \/>Version 3.6.3-0.54.2-3282-SUSE-CODE11-x86_64     <br \/>cubefileclient2:~ # uname -a     <br \/>Linux cubefileclient2 3.0.101-0.35-default #1 SMP Wed Jul 9 11:43:04 UTC 2014 (c36987d) x86_64 x86_64 x86_64 GNU\/Linux     <br \/>cubefileclient2:~ # mount.cifs -V     <br \/>mount.cifs version: 5.1 <\/p>\n<p>Ubuntu:    <br \/>root@cubefileclient:~# smbclient -V     <br \/>Version 4.1.6-Ubuntu     <br \/>root@cubefileclient:~# uname -a     <br \/>Linux cubefileclient 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU\/Linux     <br \/>root@cubefileclient:~# mount.cifs -V     <br \/>mount.cifs version: 6.0 <\/p>\n<p>So here\u2019s the solution: The Suse Enterprise 11 images contain a cifs implementation both in the kernel and in smbclient that hasn\u2019t the SMB2 protocol implemented yet. And Azure files requires SMB2 otherwise the protocol negotiation will fail.<\/p>\n<p>One closing remark: Please check the date when this was posted, software versions change all the time and what is described here may not be accurate anymore when you read this. I\u2019m not posting this to point to any specific bugs or to promote one distribution over the other. It\u2019s just a fact of life that one cannot support everything with every single version of an OS or service, this post is intended to give you ideas what to look for and give you some tools to debug low-level system behavior. And of course one could have checked the version numbers first or looked for protocol version negotiation mismatches in the debug output. But when I have no clue what to look for, I found it sometimes helpful to start with the lowest level and work my way up until I find something.&#160; <\/p>\n<p>Hope this helps,    <br \/>H.<\/p>\n<div style=\"clear:both\"><\/div>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/blogs.msdn.com\/aggbug.aspx?PostID=10558638\" width=\"1\" height=\"1\"><br \/>\nSource: msdn<\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#160; Hi! tl;dr: To mount Azure Files from linux, you need cifs support in the kernel, the right mount helper and versions recent enough to supports the SMB2 protocol version. I just got a ping from a customer who had &hellip; <a href=\"https:\/\/www.cubeos.org\/blog\/2014\/09\/linux-and-azure-files-you-might-need-some-help-here\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":60,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30],"tags":[],"class_list":["post-58","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-microsoft"],"_links":{"self":[{"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/posts\/58","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/comments?post=58"}],"version-history":[{"count":0,"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/posts\/58\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/media\/60"}],"wp:attachment":[{"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/media?parent=58"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/categories?post=58"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cubeos.org\/blog\/wp-json\/wp\/v2\/tags?post=58"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}